Web Framework Benchmarks

In the following tests, we have measured the performance of several web application platforms, full-stack frameworks, and micro-frameworks (collectively, "frameworks"). For more information, read the introduction, motivation, and latest environment details.

2018-02-14
Round 15
A valentine for performance
Roses are red
Violets are blue
A fast server
Makes users love you
Show filters panel
Showing 70 of 158 frameworks.
Test types
Hardware
JSON serialization
Single query
Multiple queries
Fortunes
Data updates
Plaintext
Physical
Cloud

JSON serialization

Best database-access responses per second, single query, Dell servers at ServerCentral(152 tests)
FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
vertx-postgres191,353
100.0%
PltJavVerNonLinRea0
ulib-postgres_fit187,519
98.0%
PltC++NonULiLinRea0
ulib-postgres182,112
95.2%
PltC++NonULiLinRea0
vertx-web-postgres176,393
92.2%
McrJavvtxNonLinRea0
ulib-mongodb175,101
91.5%
PltC++NonULiLinRea0
proteus175,048
91.5%
McrJavUtwNonLinRea0
gemini-postgres174,856
91.4%
FulJavSvtResLinRea0
revenj-jvm172,092
89.9%
FulJavSvtResLinRea0
light-4j-postgresql171,838
89.8%
PltJavLigNonLinRea0
undertow-postgresql166,597
87.1%
PltJavUtwNonLinRea0
cpoll_cppsp-postgres165,134
86.3%
PltC++NonNonLinRea0
cpoll_cppsp-postgres160,247
83.7%
PltC++NonNonLinRea0
cutelyst-thread-post156,352
81.7%
FulC++QtNonLinRea0
cutelyst-thread-post154,719
80.9%
FulC++QtNonLinRea0
ulib-mysql149,247
78.0%
PltC++NonULiLinRea0
cpoll_cppsp-raw148,204
77.5%
PltC++NonNonLinRea0
fasthttp-postgresql137,073
71.6%
PltGoNonNonLinRea0
undertow-mysql136,282
71.2%
PltJavUtwNonLinRea0
proteus-mysql134,560
70.3%
McrJavUtwNonLinRea0
jawn129,041
67.4%
FulJavSvtUtwLinRea0
undertow-mongodb128,561
67.2%
PltJavUtwNonLinRea0
go-prefork124,834
65.2%
PltGoNonNonLinRea0
cutelyst-pf-postgres117,714
61.5%
FulC++QtNonLinRea0
vertx-web-susom-post113,360
59.2%
McrJavvtxNonLinRea0
gemini-mysql113,041
59.1%
FulJavSvtResLinRea0
cutelyst-thread-mysq107,164
56.0%
FulC++QtNonLinRea0
cutelyst-thread-mysq106,255
55.5%
FulC++QtNonLinRea0
cutelyst-nginx-postg105,743
55.3%
FulC++QtngxLinRea0
kami101,111
52.8%
McrGoNonNonLinRea0
go-interpolate98,849
51.7%
PltGoNonNonLinRea0
fintrospect96,913
50.6%
McrScaNtyNonLinRea0
actframework-ebean-p96,473
50.4%
FulJavUtwNonLinRea0
jooby94,281
49.3%
FulJavNtyNonLinRea0
actframework-morphia93,612
48.9%
FulJavUtwNonLinRea0
actframework-hiberna93,110
48.7%
FulJavUtwNonLinRea0
akka-http87,715
45.8%
McrScaAkkNonLinRea0
cutelyst-pf-mysql83,682
43.7%
FulC++QtNonLinRea0
cutelyst-nginx-mysql79,649
41.6%
FulC++QtngxLinRea0
actframework-ebean-m79,635
41.6%
FulJavUtwNonLinRea0
bottle-mysql-raw77,745
40.6%
McrPyMeiNonLinRea0
grizzly-jersey73,961
38.7%
McrJavJAXNonLinRea0
treefrog-mongodb73,047
38.2%
FulC++NonNonLinRea0
actframework-hiberna70,192
36.7%
FulJavUtwNonLinRea0
silicon68,945
36.0%
McrC++NonmicLinRea0
http4s66,460
34.7%
McrScaNIOblaLinRea0
fasthttp66,424
34.7%
PltGoNonNonLinRea0
servlet-raw63,712
33.3%
PltJavSvtResLinRea0
cutelyst-pf-mysql-ep63,613
33.2%
FulC++QtNonLinRea0
wildfly-ee762,868
32.9%
FulJavSvtWilLinRea0
falcore60,414
31.6%
McrGoNonNonLinRea0
goji59,755
31.2%
McrGoNonNonLinRea0
wt-postgres59,551
31.1%
FulC++NonNonLinRea0
beego-orm-mysql59,302
31.0%
FulGoNonNonLinRea0
go-postgres59,208
30.9%
PltGoNonNonLinRea0
treefrog59,000
30.8%
FulC++NonNonLinRea0
go58,969
30.8%
PltGoNonNonLinRea0
gin58,883
30.8%
McrGoNonNonLinRea0
morepath58,150
30.4%
McrPyMeiGunLinRea0
servlet-postgres-raw57,211
29.9%
PltJavSvtResLinRea0
vertx-web-mongodb56,797
29.7%
McrJavvtxNonLinRea0
dropwizard-mysql56,521
29.5%
FulJavJAXJtyLinRea0
wt56,013
29.3%
FulC++NonNonLinRea0
undertow-jersey-hika55,937
29.2%
PltJavJAXNonLinRea0
revel-raw55,320
28.9%
FulGoNonNonLinRea0
dropwizard-postgres55,163
28.8%
FulJavJAXJtyLinRea0
undertow-mongodb-asy54,970
28.7%
PltJavUtwNonLinRea0
echo54,578
28.5%
McrGoNonNonLinRea0
revel-jet53,746
28.1%
FulGoNonNonLinRea0
revel-qbs53,019
27.7%
FulGoNonNonLinRea0
treefrog-hybrid52,684
27.5%
FulC++NonNonLinRea2
treefrog-postgres49,012
25.6%
FulC++NonNonLinRea0
play2-scala-anorm-li48,890
25.5%
FulScaNtyNonLinRea0
tornado-pypy247,638
24.9%
PltPyNonTorLinRea0
activeweb46,120
24.1%
FulJavSvtNonLinRea0
activeweb-jackson46,045
24.1%
FulJavSvtNonLinRea0
weppy-py346,018
24.0%
FulPyMeiNonLinRea0
flask-mysql-raw45,601
23.8%
McrPyMeiNonLinRea0
api_hour-mysql44,877
23.5%
McrPyasyGunLinRea0
minijax44,028
23.0%
FulJavJAXJtyLinRea0
api_hour42,563
22.2%
McrPyasyGunLinRea0
weppy42,099
22.0%
FulPyMeiNonLinRea0
play141,635
21.8%
FulJavNtyNonLinRea0
restexpress40,243
21.0%
McrJavNtyNonLinRea0
aiohttp40,097
21.0%
McrPyasyGunLinRea0
play2-scala-anorm-li38,457
20.1%
FulScaAkkNonLinRea0
dropwizard-jdbi-post36,582
19.1%
FulJavJAXJtyLinRea0
play2-java-jpa-hikar36,043
18.8%
FulJavNtyNonLinRea0
aiohttp-pg-raw35,968
18.8%
McrPyasyGunLinRea0
wicket35,055
18.3%
FulJavSvtResLinRea0
play2-java-jpa-hikar34,686
18.1%
FulJavAkkNonLinRea0
dropwizard-mongodb32,794
17.1%
FulJavJAXJtyLinRea0
tapestry32,316
16.9%
FulJavSvtResLinRea0
aah31,799
16.6%
FulGoNonNonLinRea0
undertow-jersey-c3p031,657
16.5%
PltJavJAXNonLinRea0
django-py331,229
16.3%
FulPyNonMeiLinRea0
pyramid28,486
14.9%
FulPyNonMeiLinRea0
pyramid-py228,184
14.7%
FulPyNonMeiLinRea0
wheezyweb27,585
14.4%
McrPyNonMeiLinRea0
spark25,911
13.5%
McrJavSvtResLinRea0
play2-java-jooq-hika25,736
13.4%
FulJavNtyNonLinRea0
play2-java-jooq-hika24,532
12.8%
FulJavAkkNonLinRea0
django-postgresql23,981
12.5%
FulPyNonMeiLinRea0
scruffy22,867
12.0%
McrScaUtwNonLinRea54,214
django22,507
11.8%
FulPyNonMeiLinRea0
blade22,100
11.5%
FulJavNonNonLinRea0
play2-java-ebean-hik22,022
11.5%
FulJavNtyNonLinRea0
flask-py321,640
11.3%
McrPyMeiNonLinRea0
uvicorn21,446
11.2%
PltPyNonNonLinRea0
bottle-py321,066
11.0%
McrPyMeiNonLinRea0
bottle20,751
10.8%
McrPyMeiNonLinRea0
flask20,639
10.8%
McrPyMeiNonLinRea0
play2-java-ebean-hik20,368
10.6%
FulJavAkkNonLinRea0
turbogears18,908
9.9%
McrPyNonMeiLinRea0
tornado17,015
8.9%
PltPyNonTorLinRea0
tornado-py316,362
8.6%
PltPyNonTorLinRea0
ffead-cpp-apache-mon15,902
8.3%
FulC++ffeApcLinRea0
ffead-cpp15,834
8.3%
FulC++ffeffeLinRea0
ffead-cpp-nginx-mong15,824
8.3%
FulC++ffengxLinRea0
go-mongo15,802
8.3%
PltGoNonNonLinRea0
cppcms-postgres15,751
8.2%
PltC++NonNonLinRea0
spring15,130
7.9%
FulJavUtwNonLinRea0
play2-scala-slick14,990
7.8%
FulScaAkkNonLinRea0
cppcms14,820
7.7%
PltC++NonNonLinRea0
play2-scala-slick-ne13,871
7.2%
FulScaNtyNonLinRea0
ffead-cpp-mysql12,559
6.6%
FulC++ffeffeLinRea0
ffead-cpp-postgresql12,105
6.3%
FulC++ffeffeLinRea0
ffead-cpp-nginx-post12,077
6.3%
FulC++ffengxLinRea0
ffead-cpp-apache-pos12,034
6.3%
FulC++ffeApcLinRea0
web2py-optimized11,435
6.0%
FulPyMeiNonLinRea0
cppcms-direct10,962
5.7%
PltC++NonNonLinRea0
ninja-standalone8,621
4.5%
FulJavJtyNonLinRea0
web2py8,587
4.5%
FulPyMeiNonLinRea0
tornado-postgresql-r8,176
4.3%
PltPyNonTorLinRea0
bottle-nginx-uwsgi5,066
2.6%
McrPyuWSngxLinRea0
weppy-nginx-uwsgi4,508
2.4%
FulPyuWSngxLinRea0
flask-nginx-uwsgi4,462
2.3%
McrPyNonngxLinRea0
flask-pypy2-mysql-ra4,202
2.2%
McrPyTorNonLinRea0
weppy-pypy23,496
1.8%
FulPyTorNonLinRea0
bottle-pypy22,209
1.2%
McrPyTorNonLinRea0
play2-scala-reactive1,894
1.0%
FulScaAkkNonLinRea0
play2-scala-reactive1,772
0.9%
FulScaNtyNonLinRea0
flask-pypy21,713
0.9%
McrPyNonTorLinRea0
restexpress-mysql-ra660
0.3%
McrJavNtyNonLinRea121,507
klein323
0.2%
McrPyNonTwiLinRea0
cherrypy-py3285
0.1%
McrPyNonNonLinRea0
cherrypy195
0.1%
McrPyNonNonLinRea319
ffead-cpp-apache-mys0
0.0%
FulC++ffeApcLinRea0
ffead-cpp-nginx-mysq0
0.0%
FulC++ffengxLinRea0
webware0
0.0%
McrPyNonNonLinRea1,143
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
Database-access responses per second, single query, Dell servers at ServerCentral
Framework163264128256512BestClsLngPltFEAosIAErrors
aah20,97422,41424,01927,99430,62031,79931,799FulGoNonNonLinRea0
actframework-ebean-m27,12743,98565,30079,63579,32179,15779,635FulJavUtwNonLinRea0
actframework-ebean-p30,07344,92474,31496,47389,50490,89096,473FulJavUtwNonLinRea0
actframework-hiberna22,09343,08460,10669,44970,19269,48470,192FulJavUtwNonLinRea0
actframework-hiberna27,27946,67173,54193,11088,90889,55393,110FulJavUtwNonLinRea0
actframework-morphia23,93036,36970,11090,93293,61292,49593,612FulJavUtwNonLinRea0
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
activeweb17,51132,55040,57742,97744,34546,12046,120FulJavSvtNonLinRea0
activeweb-jackson17,96433,03740,69843,43743,98646,04546,045FulJavSvtNonLinRea0
aiohttp8,60017,51030,06234,72540,09739,66640,097McrPyasyGunLinRea0
aiohttp-pg-raw10,62821,85435,14935,96834,13833,14935,968McrPyasyGunLinRea0
akka-http17,51323,09948,50387,71585,61586,52487,715McrScaAkkNonLinRea0
api_hour10,56617,73733,32539,54442,56335,04342,563McrPyasyGunLinRea0
api_hour-mysql9,68917,81230,62936,07842,09844,87744,877McrPyasyGunLinRea0
beego-orm-mysql19,00227,86947,25157,79559,30242,85459,302FulGoNonNonLinRea0
blade21,49521,30221,52821,83922,10021,10722,100FulJavNonNonLinRea0
bottle6,58512,06620,75119,39316,99216,91520,751McrPyMeiNonLinRea0
bottle-mysql-raw19,20637,45866,93676,32277,74570,17777,745McrPyMeiNonLinRea0
bottle-nginx-uwsgi4,2024,2924,5375,0664,3194,2715,066McrPyuWSngxLinRea0
bottle-py36,32612,16721,06620,37518,22918,10221,066McrPyMeiNonLinRea0
bottle-pypy21,4801,7702,2091,8881,4671,2152,209McrPyTorNonLinRea0
cpoll_cppsp-postgres46,931111,971160,247153,056117,89196,565160,247PltC++NonNonLinRea0
cpoll_cppsp-postgres41,328106,961165,134127,400155,022127,968165,134PltC++NonNonLinRea0
cpoll_cppsp-raw40,60597,389144,929148,204144,076130,261148,204PltC++NonNonLinRea0
cutelyst-nginx-mysql19,67846,43079,52379,64979,45479,10379,649FulC++QtngxLinRea0
cutelyst-nginx-postg23,87749,670105,74382,24280,97880,861105,743FulC++QtngxLinRea0
cutelyst-pf-mysql10,49318,95049,26862,00982,73983,68283,682FulC++QtNonLinRea0
cutelyst-pf-mysql-ep12,70623,40234,67960,19263,61353,77463,613FulC++QtNonLinRea0
cutelyst-pf-postgres11,51720,82132,10665,95098,344117,714117,714FulC++QtNonLinRea0
cutelyst-thread-mysq22,67059,07184,070101,966106,336107,164107,164FulC++QtNonLinRea0
cutelyst-thread-mysq26,09160,02584,368102,096105,678106,255106,255FulC++QtNonLinRea0
cutelyst-thread-post32,54376,726121,469149,807154,719153,972154,719FulC++QtNonLinRea0
cutelyst-thread-post32,29164,780112,416154,012156,352155,621156,352FulC++QtNonLinRea0
django7,68714,61822,50722,20117,52217,99222,507FulPyNonMeiLinRea0
django-postgresql7,84215,09223,98123,31219,26318,69623,981FulPyNonMeiLinRea0
django-py39,16318,27730,76131,22926,69926,33731,229FulPyNonMeiLinRea0
dropwizard-jdbi-post21,37630,74533,22736,25936,33736,58236,582FulJavJAXJtyLinRea0
dropwizard-mongodb20,23430,47431,72732,79432,27231,68232,794FulJavJAXJtyLinRea0
dropwizard-mysql20,83937,27256,52151,79547,31049,60256,521FulJavJAXJtyLinRea0
dropwizard-postgres23,18840,45455,16351,96145,74547,10355,163FulJavJAXJtyLinRea0
falcore29,02240,63060,41445,10345,53346,59960,414McrGoNonNonLinRea0
ffead-cpp14,75115,60415,83415,53115,52115,24915,834FulC++ffeffeLinRea0
ffead-cpp-apache-mon14,69315,90215,86415,64415,58915,30815,902FulC++ffeApcLinRea0
ffead-cpp-apache-mys0000000FulC++ffeApcLinRea0
ffead-cpp-apache-pos9,97612,0347,5586,6696,6086,60212,034FulC++ffeApcLinRea0
ffead-cpp-mysql10,43512,559000012,559FulC++ffeffeLinRea0
ffead-cpp-nginx-mong14,67315,82415,81015,64315,61415,76115,824FulC++ffengxLinRea0
ffead-cpp-nginx-mysq0000000FulC++ffengxLinRea0
ffead-cpp-nginx-post9,95612,0777,6456,7936,6506,64112,077FulC++ffengxLinRea0
ffead-cpp-postgresql9,94412,1057,6406,6826,6456,63912,105FulC++ffeffeLinRea0
fintrospect22,34844,45685,38595,88796,91395,62196,913McrScaNtyNonLinRea0
flask6,76212,90120,63918,13516,18015,62820,639McrPyMeiNonLinRea0
flask-mysql-raw12,31223,54340,85845,60136,06835,47645,601McrPyMeiNonLinRea0
flask-nginx-uwsgi4,0994,1724,4624,4394,2724,2014,462McrPyNonngxLinRea0
flask-py36,78813,10121,64020,94318,30118,34921,640McrPyMeiNonLinRea0
flask-pypy21,2451,6671,7131,4331,0489571,713McrPyNonTorLinRea0
flask-pypy2-mysql-ra2,6202,9614,2023,2772,4022,2704,202McrPyTorNonLinRea0
gemini-mysql34,15174,187108,185113,041109,09763,432113,041FulJavSvtResLinRea0
gemini-postgres40,426103,505167,746174,856150,425126,070174,856FulJavSvtResLinRea0
gin27,95539,92358,88346,91047,16848,55558,883McrGoNonNonLinRea0
goji25,11240,16959,75541,21125,76330,18059,755McrGoNonNonLinRea0
grizzly-jersey18,31538,78058,82872,75873,71773,96173,961McrJavJAXNonLinRea0
http4s18,17839,72964,15966,46065,41165,77066,460McrScaNIOblaLinRea0
jawn33,70744,37748,105128,935129,041125,071129,041FulJavSvtUtwLinRea0
undertow-jersey-c3p09,42721,16631,65727,81625,27725,56731,657PltJavJAXNonLinRea0
undertow-jersey-hika16,56133,90543,89755,93751,44150,83155,937PltJavJAXNonLinRea0
jooby34,27059,83985,20291,46681,71094,28194,281FulJavNtyNonLinRea0
kami27,73739,95771,522101,11192,69588,222101,111McrGoNonNonLinRea0
klein322323323322315314323McrPyNonTwiLinRea0
minijax30,57638,43641,37343,54244,02843,85544,028FulJavJAXJtyLinRea0
morepath12,44124,60047,97258,15054,86748,98158,150McrPyMeiGunLinRea0
ninja-standalone8,3898,5318,3478,4928,6218,5738,621FulJavJtyNonLinRea0
cherrypy186194192191193195195McrPyNonNonLinRea319
cherrypy-py3285285285283284282285McrPyNonNonLinRea0
cppcms14,17514,82014,53514,14614,29214,45614,820PltC++NonNonLinRea0
cppcms-direct10,47010,64110,79610,96210,68310,82610,962PltC++NonNonLinRea0
cppcms-postgres15,75115,69815,09914,55514,53514,45715,751PltC++NonNonLinRea0
echo25,89442,03754,57844,19343,63137,63954,578McrGoNonNonLinRea0
fasthttp34,83243,07766,42451,09748,46438,23466,424PltGoNonNonLinRea0
fasthttp-postgresql35,25573,18381,311103,468137,07390,577137,073PltGoNonNonLinRea0
go28,48840,56458,96948,30348,05836,92458,969PltGoNonNonLinRea0
go-interpolate28,40040,14970,19498,04298,84952,89198,849PltGoNonNonLinRea0
go-mongo15,74015,80215,68115,56915,51215,26215,802PltGoNonNonLinRea0
go-postgres27,25641,53959,20847,35046,69438,86059,208PltGoNonNonLinRea0
go-prefork33,04874,450110,412121,515124,834124,341124,834PltGoNonNonLinRea0
light-4j-postgresql40,32955,670114,170171,838170,944171,215171,838PltJavLigNonLinRea0
revel-jet22,90635,07253,74643,41744,09245,02253,746FulGoNonNonLinRea0
revel-qbs23,25136,14953,01943,66444,23745,82153,019FulGoNonNonLinRea0
revel-raw23,84837,71055,32044,96345,17146,84955,320FulGoNonNonLinRea0
servlet-postgres-raw35,78052,04152,85953,76557,21152,76657,211PltJavSvtResLinRea0
servlet-raw31,47952,01452,64453,16958,64463,71263,712PltJavSvtResLinRea0
tornado5,87910,37516,20117,01516,09815,49317,015PltPyNonTorLinRea0
tornado-postgresql-r6,8657,6327,9178,1768,0087,9638,176PltPyNonTorLinRea0
tornado-py34,9909,31313,90415,28516,36216,18316,362PltPyNonTorLinRea0
tornado-pypy25,19010,78820,96629,76244,25547,63847,638PltPyNonTorLinRea0
undertow-mongodb29,61142,79864,437125,942128,56188,554128,561PltJavUtwNonLinRea0
undertow-mongodb-asy30,21250,26848,29950,98454,97053,97854,970PltJavUtwNonLinRea0
undertow-mysql34,15947,69775,837129,063136,282133,451136,282PltJavUtwNonLinRea0
undertow-postgresql38,37547,16277,333160,148165,171166,597166,597PltJavUtwNonLinRea0
vertx-postgres38,074105,005168,967178,666183,391191,353191,353PltJavVerNonLinRea0
wildfly-ee716,14121,83826,87347,10253,40762,86862,868FulJavSvtWilLinRea0
play114,77227,66339,35441,63541,42041,18741,635FulJavNtyNonLinRea0
play2-java-ebean-hik12,03519,04819,87620,05720,17620,36820,368FulJavAkkNonLinRea0
play2-java-ebean-hik13,46622,02221,54621,57921,85421,26622,022FulJavNtyNonLinRea0
play2-java-jooq-hika14,99522,46321,06624,36723,28224,53224,532FulJavAkkNonLinRea0
play2-java-jooq-hika17,05625,71724,42325,73622,96823,18525,736FulJavNtyNonLinRea0
play2-java-jpa-hikar19,24534,19634,05334,29634,27634,68634,686FulJavAkkNonLinRea0
play2-java-jpa-hikar22,28936,00236,04335,90435,18934,71836,043FulJavNtyNonLinRea0
play2-scala-anorm-li22,53434,41037,15037,92138,45638,45738,457FulScaAkkNonLinRea0
play2-scala-anorm-li26,36547,00046,66348,89046,36347,17948,890FulScaNtyNonLinRea0
play2-scala-reactive1,6941,8071,8841,8941,8851,8581,894FulScaAkkNonLinRea0
play2-scala-reactive1,5861,7131,7181,7501,7511,7721,772FulScaNtyNonLinRea0
play2-scala-slick11,99414,99013,72614,01613,85913,59414,990FulScaAkkNonLinRea0
play2-scala-slick-ne12,98313,35013,87113,48513,50413,54713,871FulScaNtyNonLinRea0
proteus42,34658,960119,667175,048173,354173,329175,048McrJavUtwNonLinRea0
proteus-mysql36,48454,257108,610133,051133,868134,560134,560McrJavUtwNonLinRea0
pyramid8,76116,85228,48627,10424,27724,42828,486FulPyNonMeiLinRea0
pyramid-py28,94417,71128,18425,47421,40921,38228,184FulPyNonMeiLinRea0
restexpress23,51830,10940,24333,10925,56425,07440,243McrJavNtyNonLinRea0
restexpress-mysql-ra1671678316583660660McrJavNtyNonLinRea121,507
revenj-jvm45,466114,123169,879172,092149,804124,703172,092FulJavSvtResLinRea0
scruffy20,02218,84422,86713,01314,34314,30822,867McrScaUtwNonLinRea54,214
silicon31,54352,92268,05168,94566,18960,93968,945McrC++NonmicLinRea0
spark13,93625,91118,00514,90312,71111,65125,911McrJavSvtResLinRea0
spring7,88415,13013,66912,87812,76412,97515,130FulJavUtwNonLinRea0
tapestry12,39421,13628,44432,31631,90231,46232,316FulJavSvtResLinRea0
treefrog19,15035,99354,14459,00056,73850,95259,000FulC++NonNonLinRea0
treefrog-hybrid10,31618,62028,18638,53849,53852,68452,684FulC++NonNonLinRea2
treefrog-mongodb21,21729,70448,13265,98070,90273,04773,047FulC++NonNonLinRea0
treefrog-postgres15,91833,16747,44049,01242,28837,98449,012FulC++NonNonLinRea0
turbogears6,57712,55718,90816,81315,02714,72018,908McrPyNonMeiLinRea0
ulib-mongodb37,786107,107157,603175,101173,700174,404175,101PltC++NonULiLinRea0
ulib-mysql44,066106,361143,100149,247149,110148,571149,247PltC++NonULiLinRea0
ulib-postgres46,595104,798174,473176,064182,112180,491182,112PltC++NonULiLinRea0
ulib-postgres_fit40,501119,447166,642180,821187,519185,711187,519PltC++NonULiLinRea0
uvicorn12,95821,19221,44620,80619,62019,69921,446PltPyNonNonLinRea0
vertx-web-mongodb29,32853,52154,42554,38254,73556,79756,797McrJavvtxNonLinRea0
vertx-web-postgres33,13890,285141,570152,768167,888176,393176,393McrJavvtxNonLinRea0
vertx-web-susom-post26,28362,64396,592112,497113,360112,976113,360McrJavvtxNonLinRea0
web2py3,2035,9248,2768,5878,0308,3208,587FulPyMeiNonLinRea0
web2py-optimized4,0427,35311,43511,37410,40410,51311,435FulPyMeiNonLinRea0
webware0000000McrPyNonNonLinRea1,143
weppy9,56920,08036,33642,09935,34633,67842,099FulPyMeiNonLinRea0
weppy-nginx-uwsgi4,2384,3564,3994,3444,4074,5084,508FulPyuWSngxLinRea0
weppy-py39,68420,65737,56046,01843,42239,85246,018FulPyMeiNonLinRea0
weppy-pypy21,6422,3633,3453,4962,7583,0303,496FulPyTorNonLinRea0
wheezyweb8,41416,24327,58525,41122,78922,18127,585McrPyNonMeiLinRea0
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
wicket33,87134,66035,05534,30732,72732,30135,055FulJavSvtResLinRea0
wt12,36113,19526,45256,01354,67653,58856,013FulC++NonNonLinRea0
wt-postgres12,83213,98225,61059,55157,12156,70359,551FulC++NonNonLinRea0
Latency of database-access responses, Dell servers at ServerCentral
FrameworkAverage latency (lower is better)σ (SD)MaxErrors
ulib-postgres_fit2.8 ms
0.1%
1.3 ms53.9 ms0
ulib-mongodb3.0 ms
0.1%
1.4 ms57.8 ms0
light-4j-postgresql3.0 ms
0.1%
1.7 ms159.7 ms0
ulib-postgres3.0 ms
0.1%
2.1 ms121.1 ms0
proteus3.0 ms
0.1%
1.6 ms51.4 ms0
cutelyst-thread-post3.4 ms
0.1%
1.7 ms32.3 ms0
cutelyst-thread-post3.4 ms
0.1%
1.7 ms32.8 ms0
vertx-web-postgres3.4 ms
0.1%
4.5 ms62.6 ms0
ulib-mysql3.5 ms
0.1%
1.3 ms33.8 ms0
vertx-postgres3.6 ms
0.1%
6.0 ms121.3 ms0
cutelyst-pf-postgres4.4 ms
0.1%
1.9 ms26.0 ms0
proteus-mysql4.6 ms
0.1%
4.3 ms96.6 ms0
undertow-postgresql4.7 ms
0.1%
9.5 ms227.5 ms0
go-prefork4.7 ms
0.1%
10.9 ms407.5 ms0
cutelyst-thread-mysq4.8 ms
0.1%
2.2 ms39.6 ms0
cutelyst-thread-mysq4.9 ms
0.1%
2.4 ms40.4 ms0
vertx-web-susom-post5.4 ms
0.1%
8.7 ms220.1 ms0
revenj-jvm5.4 ms
0.1%
13.5 ms463.1 ms0
jooby5.9 ms
0.1%
4.2 ms60.6 ms0
cutelyst-pf-mysql6.2 ms
0.1%
2.8 ms30.1 ms0
cutelyst-nginx-postg6.3 ms
0.1%
0.8 ms42.5 ms0
cutelyst-nginx-mysql6.5 ms
0.1%
0.4 ms30.2 ms0
undertow-mysql6.7 ms
0.1%
9.2 ms188.5 ms0
actframework-ebean-p6.8 ms
0.1%
7.7 ms212.3 ms0
jawn6.9 ms
0.1%
8.6 ms128.1 ms0
fasthttp-postgresql7.0 ms
0.1%
6.8 ms82.5 ms0
actframework-hiberna7.0 ms
0.1%
8.3 ms212.9 ms0
cpoll_cppsp-postgres7.2 ms
0.1%
14.8 ms458.2 ms0
kami7.4 ms
0.1%
7.6 ms106.1 ms0
actframework-morphia7.7 ms
0.2%
8.7 ms104.0 ms0
actframework-ebean-m7.8 ms
0.2%
9.2 ms254.0 ms0
actframework-hiberna8.0 ms
0.2%
8.6 ms226.0 ms0
grizzly-jersey8.2 ms
0.2%
10.8 ms236.3 ms0
scruffy8.9 ms
0.2%
9.2 ms158.4 ms54,214
bottle-mysql-raw9.2 ms
0.2%
13.4 ms245.4 ms0
cpoll_cppsp-raw9.4 ms
0.2%
45.6 ms1150.0 ms0
go-interpolate10.4 ms
0.2%
7.7 ms88.0 ms0
dropwizard-mysql11.5 ms
0.2%
10.6 ms232.1 ms0
undertow-jersey-hika11.5 ms
0.2%
11.1 ms254.0 ms0
cutelyst-pf-mysql-ep11.9 ms
0.2%
12.3 ms181.5 ms0
morepath11.9 ms
0.2%
13.8 ms348.8 ms0
dropwizard-postgres12.1 ms
0.2%
10.5 ms174.5 ms0
revel-raw12.1 ms
0.2%
11.1 ms113.9 ms0
gin12.4 ms
0.2%
12.9 ms282.1 ms0
falcore12.6 ms
0.2%
11.0 ms133.6 ms0
beego-orm-mysql12.6 ms
0.2%
12.5 ms252.3 ms0
cpoll_cppsp-postgres12.8 ms
0.2%
71.2 ms1940.0 ms0
undertow-mongodb-asy12.8 ms
0.3%
13.6 ms272.5 ms0
revel-qbs12.9 ms
0.3%
11.5 ms125.2 ms0
tornado-pypy212.9 ms
0.3%
17.4 ms500.7 ms0
servlet-raw13.0 ms
0.3%
18.6 ms269.8 ms0
revel-jet13.0 ms
0.3%
11.4 ms115.4 ms0
api_hour-mysql13.0 ms
0.3%
21.6 ms822.1 ms0
wt13.1 ms
0.3%
39.5 ms880.3 ms0
weppy-py313.3 ms
0.3%
13.3 ms234.3 ms0
wt-postgres13.4 ms
0.3%
41.5 ms765.1 ms0
go-postgres13.9 ms
0.3%
10.0 ms95.0 ms0
play2-scala-anorm-li13.9 ms
0.3%
21.9 ms337.1 ms0
treefrog-hybrid14.1 ms
0.3%
43.9 ms918.2 ms2
treefrog-mongodb14.1 ms
0.3%
60.4 ms1640.0 ms0
echo14.3 ms
0.3%
10.3 ms99.4 ms0
go14.5 ms
0.3%
10.1 ms93.0 ms0
vertx-web-mongodb15.0 ms
0.3%
17.2 ms247.5 ms0
aiohttp15.6 ms
0.3%
17.9 ms428.2 ms0
aiohttp-pg-raw15.6 ms
0.3%
13.5 ms597.5 ms0
fintrospect16.0 ms
0.3%
92.3 ms1630.0 ms0
flask-mysql-raw16.2 ms
0.3%
17.3 ms387.0 ms0
play2-scala-anorm-li16.3 ms
0.3%
30.7 ms833.5 ms0
fasthttp16.3 ms
0.3%
19.9 ms430.3 ms0
aah16.6 ms
0.3%
12.9 ms236.7 ms0
play2-java-jpa-hikar16.6 ms
0.3%
22.2 ms445.3 ms0
activeweb16.7 ms
0.3%
25.4 ms689.6 ms0
weppy16.9 ms
0.3%
16.7 ms414.3 ms0
activeweb-jackson17.1 ms
0.3%
24.4 ms529.5 ms0
dropwizard-mongodb17.2 ms
0.3%
13.8 ms234.1 ms0
play2-java-jpa-hikar17.8 ms
0.3%
22.4 ms437.6 ms0
treefrog18.0 ms
0.4%
63.6 ms1610.0 ms0
http4s18.1 ms
0.4%
88.8 ms1640.0 ms0
servlet-postgres-raw18.5 ms
0.4%
34.6 ms412.6 ms0
goji18.8 ms
0.4%
15.8 ms148.9 ms0
wicket18.9 ms
0.4%
20.8 ms478.9 ms0
wildfly-ee719.4 ms
0.4%
30.4 ms345.3 ms0
akka-http19.9 ms
0.4%
68.1 ms1010.0 ms0
undertow-jersey-c3p020.5 ms
0.4%
9.1 ms225.7 ms0
api_hour20.8 ms
0.4%
28.2 ms458.5 ms0
restexpress20.9 ms
0.4%
29.6 ms882.2 ms0
treefrog-postgres21.1 ms
0.4%
75.3 ms1690.0 ms0
django-py321.5 ms
0.4%
20.2 ms301.2 ms0
tapestry21.7 ms
0.4%
33.3 ms1020.0 ms0
minijax22.7 ms
0.4%
92.8 ms1650.0 ms0
pyramid23.4 ms
0.5%
22.2 ms397.9 ms0
play124.7 ms
0.5%
102.7 ms1610.0 ms0
wheezyweb25.0 ms
0.5%
22.5 ms430.3 ms0
play2-java-ebean-hik25.3 ms
0.5%
16.9 ms248.0 ms0
dropwizard-jdbi-post26.1 ms
0.5%
65.4 ms595.6 ms0
blade26.3 ms
0.5%
21.4 ms256.7 ms0
pyramid-py226.5 ms
0.5%
24.9 ms474.4 ms0
play2-java-ebean-hik28.6 ms
0.6%
40.9 ms872.1 ms0
django-postgresql29.2 ms
0.6%
25.9 ms619.5 ms0
bottle30.4 ms
0.6%
25.0 ms470.5 ms0
flask-py330.4 ms
0.6%
26.5 ms458.9 ms0
uvicorn30.6 ms
0.6%
33.8 ms587.4 ms0
bottle-py330.7 ms
0.6%
27.6 ms694.4 ms0
tornado-py331.3 ms
0.6%
16.2 ms405.0 ms0
django31.7 ms
0.6%
29.7 ms575.7 ms0
ffead-cpp-nginx-mong32.5 ms
0.6%
3.5 ms267.8 ms0
tornado33.1 ms
0.6%
20.1 ms584.2 ms0
ffead-cpp-apache-mon33.1 ms
0.6%
2.5 ms63.3 ms0
go-mongo33.5 ms
0.7%
3.9 ms86.9 ms0
ffead-cpp33.6 ms
0.7%
3.5 ms270.9 ms0
flask34.2 ms
0.7%
28.0 ms586.2 ms0
cppcms35.4 ms
0.7%
3.5 ms43.8 ms0
cppcms-postgres35.4 ms
0.7%
4.5 ms45.3 ms0
cherrypy-py335.4 ms
0.7%
6.8 ms278.1 ms0
turbogears37.4 ms
0.7%
32.8 ms462.3 ms0
silicon38.3 ms
0.7%
228.2 ms3380.0 ms0
spring40.3 ms
0.8%
21.1 ms198.6 ms0
cppcms-direct46.6 ms
0.9%
5.6 ms249.4 ms0
undertow-mongodb48.8 ms
1.0%
115.7 ms787.5 ms0
spark48.8 ms
1.0%
44.6 ms684.4 ms0
gemini-mysql49.6 ms
1.0%
99.1 ms1090.0 ms0
web2py-optimized50.8 ms
1.0%
38.5 ms701.1 ms0
cherrypy51.0 ms
1.0%
11.3 ms271.8 ms319
ninja-standalone57.7 ms
1.1%
32.5 ms878.0 ms0
play2-java-jooq-hika60.9 ms
1.2%
167.2 ms1120.0 ms0
tornado-postgresql-r62.0 ms
1.2%
13.7 ms268.6 ms0
web2py66.6 ms
1.3%
55.4 ms681.0 ms0
ffead-cpp-postgresql75.1 ms
1.5%
11.1 ms291.2 ms0
ffead-cpp-nginx-post76.9 ms
1.5%
4.3 ms289.1 ms0
ffead-cpp-apache-pos77.5 ms
1.5%
9.0 ms295.4 ms0
play2-java-jooq-hika96.8 ms
1.9%
277.2 ms1710.0 ms0
play2-scala-slick-ne108.9 ms
2.1%
147.8 ms1100.0 ms0
weppy-nginx-uwsgi113.5 ms
2.2%
13.4 ms344.5 ms0
bottle-nginx-uwsgi119.4 ms
2.3%
17.9 ms342.8 ms0
flask-nginx-uwsgi121.8 ms
2.4%
16.1 ms356.7 ms0
gemini-postgres156.4 ms
3.1%
299.2 ms1650.0 ms0
play2-scala-slick175.2 ms
3.4%
556.5 ms6680.0 ms0
weppy-pypy2197.0 ms
3.9%
197.6 ms2630.0 ms0
flask-pypy2-mysql-ra231.4 ms
4.5%
174.9 ms1930.0 ms0
restexpress-mysql-ra278.1 ms
5.4%
607.2 ms3340.0 ms121,507
play2-scala-reactive334.0 ms
6.5%
331.7 ms1300.0 ms0
play2-scala-reactive393.2 ms
7.7%
443.6 ms1880.0 ms0
bottle-pypy2413.4 ms
8.1%
295.1 ms3230.0 ms0
flask-pypy2519.2 ms
10.2%
317.6 ms2470.0 ms0
klein1390.0 ms
27.2%
693.6 ms5800.0 ms0
webware5110.0 ms
100.0%
1520.0 ms7000.0 ms1,143
Framework overhead of database-access responses, Dell servers at ServerCentral
Framework bestvsPlatform bestRatio (higher is better)
tornado-pypy247,638vstornado17,015
280.0%
fasthttp-postgresql137,073vsgo58,969
232.4%
go-prefork124,834vsgo58,969
211.7%
undertow-jersey-hika55,937vsundertow-jersey-c3p031,657
176.7%
kami101,111vsgo58,969
171.5%
go-interpolate98,849vsgo58,969
167.6%
treefrog-mongodb73,047vstreefrog59,000
123.8%
fasthttp66,424vsgo58,969
112.6%
wt-postgres59,551vswt56,013
106.3%
falcore60,414vsgo58,969
102.5%
goji59,755vsgo58,969
101.3%
beego-orm-mysql59,302vsgo58,969
100.6%
go-postgres59,208vsgo58,969
100.4%
gin58,883vsgo58,969
99.9%
dropwizard-postgres55,163vsservlet-postgres-raw57,211
96.4%
tornado-py316,362vstornado17,015
96.2%
revel-raw55,320vsgo58,969
93.8%
echo54,578vsgo58,969
92.6%
revel-jet53,746vsgo58,969
91.1%
revel-qbs53,019vsgo58,969
89.9%
servlet-postgres-raw57,211vsservlet-raw63,712
89.8%
treefrog-hybrid52,684vstreefrog59,000
89.3%
dropwizard-mysql56,521vsservlet-raw63,712
88.7%
dropwizard-jdbi-post36,582vsservlet-postgres-raw57,211
63.9%
undertow-jersey-c3p031,657vsundertow-jersey-hika55,937
56.6%
aah31,799vsgo58,969
53.9%
tornado-postgresql-r8,176vstornado17,015
48.1%
play2-scala-anorm-li38,457vsakka-http87,715
43.8%
play2-java-jpa-hikar34,686vsakka-http87,715
39.5%
play2-java-jooq-hika24,532vsakka-http87,715
28.0%
go-mongo15,802vsgo58,969
26.8%
play2-java-ebean-hik20,368vsakka-http87,715
23.2%
play2-scala-slick14,990vsakka-http87,715
17.1%
play2-scala-reactive1,894vsakka-http87,715
2.2%
Best (bar chart)
Data table
Latency
Framework overhead
Best (bar chart)
Data table
Latency
Framework overhead

Requirements summary

In this test, each request is processed by fetching a single row from a simple database table. That row is then serialized as a JSON response.

Example response:

HTTP/1.1 200 OK Content-Length: 32 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"id":3217,"randomNumber":2149}

For a more detailed description of the requirements, see the Source Code and Requirements section.

Responses per second at 20 queries per request, Dell servers at ServerCentral(149 tests)
FrameworkPerformance (higher is better)ClsLngPltFEAosIAErrors
undertow-postgresql9,514
100.0%(92.0%)
PltJavUtwNonLinRea0
wt-postgres9,349
98.3%(90.4%)
FulC++NonNonLinRea0
jooby9,196
96.7%(88.9%)
FulJavNtyNonLinRea0
ulib-postgres_fit9,171
96.4%(88.7%)
PltC++NonULiLinRea0
vertx-web-susom-post9,130
96.0%(88.3%)
McrJavvtxNonLinRea0
vertx-web-postgres9,012
94.7%(87.2%)
McrJavvtxNonLinRea0
light-4j-postgresql8,855
93.1%(85.6%)
PltJavLigNonLinRea0
ulib-mongodb8,745
91.9%(84.6%)
PltC++NonULiLinRea0
ulib-postgres8,699
91.4%(84.1%)
PltC++NonULiLinRea0
vertx-postgres8,697
91.4%(84.1%)
PltJavVerNonLinRea0
jawn8,332
87.6%(80.6%)
FulJavSvtUtwLinRea0
cutelyst-thread-post7,808
82.1%(75.5%)
FulC++QtNonLinRea0
cutelyst-thread-post7,738
81.3%(74.8%)
FulC++QtNonLinRea0
cutelyst-nginx-postg7,630
80.2%(73.8%)
FulC++QtngxLinRea0
cutelyst-pf-postgres7,563
79.5%(73.1%)
FulC++QtNonLinRea0
undertow-mysql7,286
76.6%(70.5%)
PltJavUtwNonLinRea0
ulib-mysql7,187
75.5%(69.5%)
PltC++NonULiLinRea0
cpoll_cppsp-postgres7,177
75.4%(69.4%)
PltC++NonNonLinRea0
treefrog-mongodb6,688
70.3%(64.7%)
FulC++NonNonLinRea0
servlet-postgres-raw6,341
66.6%(61.3%)
PltJavSvtResLinRea0
cpoll_cppsp-raw6,333
66.6%(61.2%)
PltC++NonNonLinRea0
cpoll_cppsp-postgres6,197
65.1%(59.9%)
PltC++NonNonLinRea0
revenj-jvm6,133
64.5%(59.3%)
FulJavSvtResLinRea0
gemini-postgres6,109
64.2%(59.1%)
FulJavSvtResLinRea0
fasthttp-postgresql6,091
64.0%(58.9%)
PltGoNonNonLinRea0
go-prefork6,072
63.8%(58.7%)
PltGoNonNonLinRea0
silicon6,004
63.1%(58.1%)
McrC++NonmicLinRea53
akka-http5,897
62.0%(57.0%)
McrScaAkkNonLinRea0
bottle-mysql-raw5,737
60.3%(55.5%)
McrPyMeiNonLinRea0
flask-mysql-raw5,555
58.4%(53.7%)
McrPyMeiNonLinRea0
aiohttp-pg-raw5,443
57.2%(52.6%)
McrPyasyGunLinRea0
wt5,433
57.1%(52.5%)
FulC++NonNonLinRea0
cutelyst-thread-mysq5,368
56.4%(51.9%)
FulC++QtNonLinRea0
http4s5,366
56.4%(51.9%)
McrScaNIOblaLinRea0
cutelyst-thread-mysq5,336
56.1%(51.6%)
FulC++QtNonLinRea0
wildfly-ee75,292
55.6%(51.2%)
FulJavSvtWilLinRea0
api_hour-mysql5,189
54.5%(50.2%)
McrPyasyGunLinRea0
fintrospect5,168
54.3%(50.0%)
McrScaNtyNonLinRea0
servlet-raw5,100
53.6%(49.3%)
PltJavSvtResLinRea0
kami4,938
51.9%(47.8%)
McrGoNonNonLinRea0
actframework-hiberna4,812
50.6%(46.5%)
FulJavUtwNonLinRea0
cppcms4,739
49.8%(45.8%)
PltC++NonNonLinRea0
web2py-optimized4,712
49.5%(45.6%)
FulPyMeiNonLinRea0
cppcms-direct4,707
49.5%(45.5%)
PltC++NonNonLinRea0
cutelyst-nginx-mysql4,696
49.4%(45.4%)
FulC++QtngxLinRea0
actframework-morphia4,672
49.1%(45.2%)
FulJavUtwNonLinRea0
cutelyst-pf-mysql4,659
49.0%(45.1%)
FulC++QtNonLinRea0
play14,640
48.8%(44.9%)
FulJavNtyNonLinRea0
actframework-ebean-p4,609
48.4%(44.6%)
FulJavUtwNonLinRea0
cutelyst-pf-mysql-ep4,601
48.4%(44.5%)
FulC++QtNonLinRea0
morepath4,247
44.6%(41.1%)
McrPyMeiGunLinRea0
actframework-hiberna4,236
44.5%(41.0%)
FulJavUtwNonLinRea0
actframework-ebean-m4,046
42.5%(39.1%)
FulJavUtwNonLinRea0
grizzly-jersey3,878
40.8%(37.5%)
McrJavJAXNonLinRea0
weppy-py33,843
40.4%(37.2%)
FulPyMeiNonLinRea0
undertow-jersey-hika3,652
38.4%(35.3%)
PltJavJAXNonLinRea0
ninja-standalone3,629
38.1%(35.1%)
FulJavJtyNonLinRea0
go-interpolate3,467
36.4%(33.5%)
PltGoNonNonLinRea0
api_hour3,458
36.3%(33.4%)
McrPyasyGunLinRea0
tornado-pypy23,438
36.1%(33.2%)
PltPyNonTorLinRea0
tapestry3,204
33.7%(31.0%)
FulJavSvtResLinRea0
weppy-nginx-uwsgi3,173
33.4%(30.7%)
FulPyuWSngxLinRea0
minijax3,135
33.0%(30.3%)
FulJavJAXJtyLinRea0
dropwizard-postgres3,092
32.5%(29.9%)
FulJavJAXJtyLinRea0
weppy3,029
31.8%(29.3%)
FulPyMeiNonLinRea0
dropwizard-mysql2,970
31.2%(28.7%)
FulJavJAXJtyLinRea0
gemini-mysql2,910
30.6%(28.1%)
FulJavSvtResLinRea0
undertow-mongodb-asy2,904
30.5%(28.1%)
PltJavUtwNonLinRea266
gin2,870
30.2%(27.8%)
McrGoNonNonLinRea0
goji2,850
30.0%(27.6%)
McrGoNonNonLinRea0
vertx-web-mongodb2,845
29.9%(27.5%)
McrJavvtxNonLinRea12
play2-scala-anorm-li2,826
29.7%(27.3%)
FulScaAkkNonLinRea0
falcore2,797
29.4%(27.1%)
McrGoNonNonLinRea0
play2-scala-anorm-li2,796
29.4%(27.0%)
FulScaNtyNonLinRea0
cppcms-postgres2,782
29.2%(26.9%)
PltC++NonNonLinRea0
revel-raw2,716
28.5%(26.3%)
FulGoNonNonLinRea0
web2py2,699
28.4%(26.1%)
FulPyMeiNonLinRea0
revel-qbs2,603
27.4%(25.2%)
FulGoNonNonLinRea0
revel-jet2,580
27.1%(25.0%)
FulGoNonNonLinRea0
wicket2,559
26.9%(24.7%)
FulJavSvtResLinRea0
treefrog-hybrid2,497
26.2%(24.1%)
FulC++NonNonLinRea2
flask-py32,485
26.1%(24.0%)
McrPyMeiNonLinRea0
undertow-mongodb2,483
26.1%(24.0%)
PltJavUtwNonLinRea10,091
activeweb-jackson2,443
25.7%(23.6%)
FulJavSvtNonLinRea0
aiohttp2,372
24.9%(22.9%)
McrPyasyGunLinRea0
fasthttp2,350
24.7%(22.7%)
PltGoNonNonLinRea0
play2-java-jpa-hikar2,311
24.3%(22.4%)
FulJavAkkNonLinRea0
go2,279
24.0%(22.0%)
PltGoNonNonLinRea0
play2-java-jpa-hikar2,271
23.9%(22.0%)
FulJavNtyNonLinRea0
go-postgres2,265
23.8%(21.9%)
PltGoNonNonLinRea0
flask-nginx-uwsgi2,247
23.6%(21.7%)
McrPyNonngxLinRea0
echo2,247
23.6%(21.7%)
McrGoNonNonLinRea0
activeweb2,245
23.6%(21.7%)
FulJavSvtNonLinRea0
aah2,221
23.3%(21.5%)
FulGoNonNonLinRea0
treefrog2,216
23.3%(21.4%)
FulC++NonNonLinRea0
dropwizard-jdbi-post2,108
22.2%(20.4%)
FulJavJAXJtyLinRea102
flask2,074
21.8%(20.1%)
McrPyMeiNonLinRea0
django-py32,037
21.4%(19.7%)
FulPyNonMeiLinRea0
play2-java-jooq-hika2,028
21.3%(19.6%)
FulJavNtyNonLinRea0
uvicorn1,985
20.9%(19.2%)
PltPyNonNonLinRea0
bottle-nginx-uwsgi1,839
19.3%(17.8%)
McrPyuWSngxLinRea0
treefrog-postgres1,830
19.2%(17.7%)
FulC++NonNonLinRea0
bottle-py31,750
18.4%(16.9%)
McrPyMeiNonLinRea0
dropwizard-mongodb1,741
18.3%(16.8%)
FulJavJAXJtyLinRea15,783
wheezyweb1,734
18.2%(16.8%)
McrPyNonMeiLinRea0
pyramid1,733
18.2%(16.8%)
FulPyNonMeiLinRea0
django-postgresql1,714
18.0%(16.6%)
FulPyNonMeiLinRea0
bottle1,623
17.1%(15.7%)
McrPyMeiNonLinRea0
django1,605
16.9%(15.5%)
FulPyNonMeiLinRea0
turbogears1,593
16.7%(15.4%)
McrPyNonMeiLinRea0
tornado-py31,589
16.7%(15.4%)
PltPyNonTorLinRea0
weppy-pypy21,585
16.7%(15.3%)
FulPyTorNonLinRea0
restexpress1,526
16.0%(14.8%)
McrJavNtyNonLinRea0
spring1,516
15.9%(14.7%)
FulJavUtwNonLinRea0
flask-pypy2-mysql-ra1,506
15.8%(14.6%)
McrPyTorNonLinRea0
tornado1,501
15.8%(14.5%)
PltPyNonTorLinRea0
pyramid-py21,449
15.2%(14.0%)
FulPyNonMeiLinRea0
play2-java-ebean-hik1,272
13.4%(12.3%)
FulJavAkkNonLinRea0
play2-java-ebean-hik1,258
13.2%(12.2%)
FulJavNtyNonLinRea0
play2-scala-reactive1,250
13.1%(12.1%)
FulScaNtyNonLinRea0
play2-java-jooq-hika1,241
13.0%(12.0%)
FulJavAkkNonLinRea0
blade1,111
11.7%(10.7%)
FulJavNonNonLinRea0
beego-orm-mysql1,087
11.4%(10.5%)
FulGoNonNonLinRea0
play2-scala-reactive967
10.2%(9.4%)
FulScaAkkNonLinRea0
go-mongo785
8.3%(7.6%)
PltGoNonNonLinRea0
play2-scala-slick689
7.2%(6.7%)
FulScaAkkNonLinRea247
play2-scala-slick-ne651
6.8%(6.3%)
FulScaNtyNonLinRea0
restexpress-mysql-ra516
5.4%(5.0%)
McrJavNtyNonLinRea76,363
undertow-jersey-c3p0454
4.8%(4.4%)
PltJavJAXNonLinRea0
spark384
4.0%(3.7%)
McrJavSvtResLinRea0
tornado-postgresql-r376
4.0%(3.6%)
PltPyNonTorLinRea911
ffead-cpp-apache-pos363
3.8%(3.5%)
FulC++ffeApcLinRea0
ffead-cpp-nginx-post362
3.8%(3.5%)
FulC++ffengxLinRea0
bottle-pypy2283
3.0%(2.7%)
McrPyTorNonLinRea0
flask-pypy2274
2.9%(2.6%)
McrPyNonTorLinRea0
cherrypy-py331
0.3%(0.3%)
McrPyNonNonLinRea366
cherrypy25
0.3%(0.2%)
McrPyNonNonLinRea218
klein20
0.2%(0.2%)
McrPyNonTwiLinRea443
ffead-cpp0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-apache-mon0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-apache-mys0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-mysql0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-nginx-mong0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-nginx-mysq0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-postgresql0
0.0%(0.0%)
FulC++ffeffeLinRea0
webware0
0.0%(0.0%)
McrPyNonNonLinRea807
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
Responses per second, multiple queries, Dell servers at ServerCentral
Framework15101520ClsLngPltFEAosIAErrors
aah31,8928,3954,1452,9282,221FulGoNonNonLinRea0
actframework-ebean-m79,04716,2918,0955,3024,046FulJavUtwNonLinRea0
actframework-ebean-p91,51618,4459,2706,2104,609FulJavUtwNonLinRea0
actframework-hiberna68,96016,4898,5395,6974,236FulJavUtwNonLinRea0
actframework-hiberna90,45518,9989,4836,3864,812FulJavUtwNonLinRea0
actframework-morphia88,46118,0039,1166,1384,672FulJavUtwNonLinRea0
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
activeweb14,8867,6054,2822,9672,245FulJavSvtNonLinRea0
activeweb-jackson46,6459,7504,9023,2682,443FulJavSvtNonLinRea0
aiohttp40,2108,8774,8003,0932,372McrPyasyGunLinRea0
aiohttp-pg-raw33,17215,9649,6766,9565,443McrPyasyGunLinRea0
akka-http82,71821,66011,1967,7415,897McrScaAkkNonLinRea0
api_hour35,03612,3006,1374,4153,458McrPyasyGunLinRea0
api_hour-mysql42,65016,6439,3966,5865,189McrPyasyGunLinRea0
beego-orm-mysql16,6864,0472,1481,4921,087FulGoNonNonLinRea0
blade21,5264,3642,1791,4681,111FulJavNonNonLinRea0
bottle16,0495,4953,1002,1601,623McrPyMeiNonLinRea0
bottle-mysql-raw64,86121,37811,3587,6495,737McrPyMeiNonLinRea0
bottle-nginx-uwsgi4,8805,6883,3402,3671,839McrPyuWSngxLinRea0
bottle-py317,5695,9253,3612,3251,750McrPyMeiNonLinRea0
bottle-pypy22,007632376322283McrPyTorNonLinRea0
cpoll_cppsp-postgres95,01525,47913,5159,3157,177PltC++NonNonLinRea0
cpoll_cppsp-postgres127,80525,20512,4728,2596,197PltC++NonNonLinRea0
cpoll_cppsp-raw130,47425,68912,7998,4836,333PltC++NonNonLinRea0
cutelyst-nginx-mysql78,64018,1699,3266,2454,696FulC++QtngxLinRea0
cutelyst-nginx-postg82,59629,21314,95710,0897,630FulC++QtngxLinRea0
cutelyst-pf-mysql82,88018,3729,2936,2044,659FulC++QtNonLinRea0
cutelyst-pf-mysql-ep53,52717,5549,0876,1194,601FulC++QtNonLinRea0
cutelyst-pf-postgres113,87929,58015,01910,0827,563FulC++QtNonLinRea0
cutelyst-thread-mysq106,77221,48410,7437,1605,368FulC++QtNonLinRea0
cutelyst-thread-mysq106,01121,50510,7337,1535,336FulC++QtNonLinRea0
cutelyst-thread-post154,90631,44515,65010,3837,738FulC++QtNonLinRea0
cutelyst-thread-post156,04731,54015,61210,4307,808FulC++QtNonLinRea0
django16,8885,4903,0062,0411,605FulPyNonMeiLinRea0
django-postgresql17,1815,9893,1642,2561,714FulPyNonMeiLinRea0
django-py324,3377,2853,8792,6722,037FulPyNonMeiLinRea0
dropwizard-jdbi-post42,6418,9344,6782,8532,108FulJavJAXJtyLinRea102
dropwizard-mongodb31,4017,9673,8932,3291,741FulJavJAXJtyLinRea15,783
dropwizard-mysql44,93313,1526,0913,9512,970FulJavJAXJtyLinRea0
dropwizard-postgres44,31813,9846,4444,0283,092FulJavJAXJtyLinRea0
falcore46,97111,0265,6033,7282,797McrGoNonNonLinRea0
ffead-cpp00000FulC++ffeffeLinRea0
ffead-cpp-apache-mon00000FulC++ffeApcLinRea0
ffead-cpp-apache-mys00000FulC++ffeApcLinRea0
ffead-cpp-apache-pos6,8831,455730489363FulC++ffeApcLinRea0
ffead-cpp-mysql00000FulC++ffeffeLinRea0
ffead-cpp-nginx-mong00000FulC++ffengxLinRea0
ffead-cpp-nginx-mysq00000FulC++ffengxLinRea0
ffead-cpp-nginx-post6,8441,441724485362FulC++ffengxLinRea0
ffead-cpp-postgresql00000FulC++ffeffeLinRea0
fintrospect90,61219,84310,1926,8275,168McrScaNtyNonLinRea0
flask15,1596,2043,7512,6192,074McrPyMeiNonLinRea0
flask-mysql-raw31,36616,9199,7127,1145,555McrPyMeiNonLinRea0
flask-nginx-uwsgi4,6075,7193,9342,8312,247McrPyNonngxLinRea0
flask-py317,1807,1184,2533,1352,485McrPyMeiNonLinRea0
flask-pypy21,321557334276274McrPyNonTorLinRea0
flask-pypy2-mysql-ra2,5921,6671,4001,5981,506McrPyTorNonLinRea0
gemini-mysql59,44711,7756,0974,3082,910FulJavSvtResLinRea0
gemini-postgres114,44125,50011,8448,8816,109FulJavSvtResLinRea0
gin46,46411,3435,4773,8462,870McrGoNonNonLinRea0
goji22,42811,1835,6003,6602,850McrGoNonNonLinRea0
grizzly-jersey72,61615,5347,7695,1473,878McrJavJAXNonLinRea0
http4s61,88519,99110,6257,1125,366McrScaNIOblaLinRea0
jawn117,53131,12016,19910,9808,332FulJavSvtUtwLinRea0
undertow-jersey-c3p025,3795,6022,9201,802454PltJavJAXNonLinRea0
undertow-jersey-hika51,04815,4167,6405,1493,652PltJavJAXNonLinRea0
jooby92,81136,14418,11812,2079,196FulJavNtyNonLinRea0
kami84,21120,4209,9476,5124,938McrGoNonNonLinRea0
klein302108583120McrPyNonTwiLinRea443
minijax43,81411,8666,0174,1113,135FulJavJAXJtyLinRea0
morepath44,48915,8818,3135,6524,247McrPyMeiGunLinRea0
ninja-standalone8,9857,4496,4984,8383,629FulJavJtyNonLinRea0
cherrypy201101583625McrPyNonNonLinRea218
cherrypy-py3275124734831McrPyNonNonLinRea366
cppcms14,34211,5589,2536,3604,739PltC++NonNonLinRea0
cppcms-direct10,6049,0757,7266,3754,707PltC++NonNonLinRea0
cppcms-postgres14,85611,8069,1495,1052,782PltC++NonNonLinRea0
echo37,2287,8954,2262,9952,247McrGoNonNonLinRea0
fasthttp38,4488,3144,5323,1272,350PltGoNonNonLinRea0
fasthttp-postgresql90,96522,96111,9017,9646,091PltGoNonNonLinRea0
go37,8418,1914,3853,0112,279PltGoNonNonLinRea0
go-interpolate56,35113,2847,1034,6073,467PltGoNonNonLinRea0
go-mongo15,2993,2481,6091,063785PltGoNonNonLinRea0
go-postgres38,2167,8484,3642,9462,265PltGoNonNonLinRea0
go-prefork122,27524,52112,1598,0656,072PltGoNonNonLinRea0
light-4j-postgresql172,23835,78917,81811,8678,855PltJavLigNonLinRea0
revel-jet45,48710,1615,0763,4032,580FulGoNonNonLinRea0
revel-qbs45,53210,3565,2193,4372,603FulGoNonNonLinRea0
revel-raw45,81210,8115,4693,6962,716FulGoNonNonLinRea0
servlet-postgres-raw51,88624,89012,6008,4396,341PltJavSvtResLinRea0
servlet-raw50,55020,37610,2796,8495,100PltJavSvtResLinRea0
tornado14,8525,2152,8821,9661,501PltPyNonTorLinRea0
tornado-postgresql-r7,4044,2012,584733376PltPyNonTorLinRea911
tornado-py315,7085,9783,2002,1091,589PltPyNonTorLinRea0
tornado-pypy245,10912,4046,7884,5673,438PltPyNonTorLinRea0
undertow-mongodb65,51210,1095,0993,7602,483PltJavUtwNonLinRea10,091
undertow-mongodb-asy54,47012,0125,9934,0062,904PltJavUtwNonLinRea266
undertow-mysql135,38529,18614,5469,7717,286PltJavUtwNonLinRea0
undertow-postgresql159,35938,03618,97112,6549,514PltJavUtwNonLinRea0
vertx-postgres187,90333,86216,87711,3688,697PltJavVerNonLinRea0
wildfly-ee763,85323,12311,0667,5975,292FulJavSvtWilLinRea0
play141,14214,8448,5716,0324,640FulJavNtyNonLinRea0
play2-java-ebean-hik20,3364,9372,5201,6941,272FulJavAkkNonLinRea0
play2-java-ebean-hik20,3204,7842,4901,6611,258FulJavNtyNonLinRea0
play2-java-jooq-hika22,4707,2172,8232,5061,241FulJavAkkNonLinRea0
play2-java-jooq-hika22,5276,1483,8681,6722,028FulJavNtyNonLinRea0
play2-java-jpa-hikar33,4178,8674,5133,0562,311FulJavAkkNonLinRea0
play2-java-jpa-hikar34,1648,4634,4973,0102,271FulJavNtyNonLinRea0
play2-scala-anorm-li39,95711,1165,7063,7282,826FulScaAkkNonLinRea0
play2-scala-anorm-li44,44210,7455,4983,7472,796FulScaNtyNonLinRea0
play2-scala-reactive14,3173,7992,0071,382967FulScaAkkNonLinRea0
play2-scala-reactive15,8863,1872,3261,7001,250FulScaNtyNonLinRea0
play2-scala-slick13,7402,6871,367927689FulScaAkkNonLinRea247
play2-scala-slick-ne13,3952,6791,336889651FulScaNtyNonLinRea0
pyramid22,6676,2603,3712,2661,733FulPyNonMeiLinRea0
pyramid-py219,2635,4202,8851,9661,449FulPyNonMeiLinRea0
restexpress24,9925,3842,6091,7991,526McrJavNtyNonLinRea0
restexpress-mysql-ra666666333636516McrJavNtyNonLinRea76,363
revenj-jvm122,58324,73812,3518,0856,133FulJavSvtResLinRea0
silicon60,57120,71911,4117,8806,004McrC++NonmicLinRea53
spark12,1405,5413,281524384McrJavSvtResLinRea0
spring30,3866,1833,0902,0641,516FulJavUtwNonLinRea0
tapestry31,13111,0156,1114,1433,204FulJavSvtResLinRea0
treefrog45,3589,0184,4972,9972,216FulC++NonNonLinRea0
treefrog-hybrid51,31010,1495,0323,3342,497FulC++NonNonLinRea2
treefrog-mongodb133,73526,97513,4348,9086,688FulC++NonNonLinRea0
treefrog-postgres37,9947,4453,7202,4521,830FulC++NonNonLinRea0
turbogears14,3825,1892,9602,0521,593McrPyNonMeiLinRea0
ulib-mongodb174,52435,06717,53911,6768,745PltC++NonULiLinRea0
ulib-mysql149,22329,04514,3849,6257,187PltC++NonULiLinRea0
ulib-postgres179,92335,36217,52511,6608,699PltC++NonULiLinRea0
ulib-postgres_fit185,38237,10818,44012,1949,171PltC++NonULiLinRea0
uvicorn20,5646,8613,7572,6011,985PltPyNonNonLinRea0
vertx-web-mongodb55,74911,7095,7313,8122,845McrJavvtxNonLinRea12
vertx-web-postgres169,45435,65717,86711,9809,012McrJavvtxNonLinRea0
vertx-web-susom-post112,20533,21417,69812,0979,130McrJavvtxNonLinRea0
web2py8,0165,5124,0723,2342,699FulPyMeiNonLinRea0
web2py-optimized14,80610,6587,5905,9374,712FulPyMeiNonLinRea0
webware00000McrPyNonNonLinRea807
weppy32,08310,3785,7844,1223,029FulPyMeiNonLinRea0
weppy-nginx-uwsgi4,4825,8555,5624,1813,173FulPyuWSngxLinRea0
weppy-py336,80611,8336,8615,0063,843FulPyMeiNonLinRea0
weppy-pypy24,4042,3281,8751,7811,585FulPyTorNonLinRea0
wheezyweb27,2426,4823,3352,2701,734McrPyNonMeiLinRea0
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
wicket30,3899,3134,9553,4472,559FulJavSvtResLinRea0
wt54,02521,71210,8597,2205,433FulC++NonNonLinRea0
wt-postgres56,54737,68818,80112,4839,349FulC++NonNonLinRea0
Latency of 20-query responses, Dell servers at ServerCentral
FrameworkAverage latency (lower is better)σ (SD)MaxErrors
undertow-postgresql53.5 ms
0.8%
33.9 ms345.7 ms0
jooby54.2 ms
0.8%
12.5 ms273.9 ms0
vertx-web-susom-post55.5 ms
0.9%
12.2 ms173.1 ms0
vertx-web-postgres56.2 ms
0.9%
22.0 ms210.4 ms0
ulib-postgres_fit56.2 ms
0.9%
23.3 ms209.4 ms0
light-4j-postgresql57.7 ms
0.9%
3.8 ms166.3 ms0
vertx-postgres58.0 ms
0.9%
34.6 ms535.9 ms0
ulib-mongodb59.0 ms
0.9%
29.3 ms172.3 ms0
ulib-postgres59.2 ms
0.9%
29.4 ms218.6 ms0
jawn61.1 ms
0.9%
37.1 ms403.1 ms0
minijax62.2 ms
1.0%
28.6 ms688.6 ms0
wt-postgres63.2 ms
1.0%
101.5 ms2050.0 ms0
cutelyst-thread-post66.8 ms
1.0%
32.1 ms272.6 ms0
cutelyst-nginx-postg67.0 ms
1.0%
2.4 ms82.6 ms0
cutelyst-thread-post67.3 ms
1.0%
32.7 ms367.1 ms0
cutelyst-pf-postgres68.5 ms
1.1%
31.7 ms274.7 ms0
undertow-mysql70.0 ms
1.1%
40.1 ms463.5 ms0
ulib-mysql71.1 ms
1.1%
27.3 ms181.7 ms0
cpoll_cppsp-postgres71.5 ms
1.1%
25.0 ms602.9 ms0
servlet-postgres-raw77.8 ms
1.2%
24.9 ms557.4 ms0
go-prefork79.4 ms
1.2%
34.1 ms492.6 ms0
revenj-jvm80.4 ms
1.2%
25.1 ms534.3 ms0
fasthttp-postgresql81.4 ms
1.3%
28.8 ms367.3 ms0
treefrog-mongodb81.7 ms
1.3%
71.6 ms1090.0 ms0
akka-http82.5 ms
1.3%
24.7 ms501.6 ms0
cpoll_cppsp-postgres83.0 ms
1.3%
19.8 ms724.3 ms0
bottle-mysql-raw84.2 ms
1.3%
27.1 ms285.2 ms0
flask-mysql-raw86.7 ms
1.3%
39.7 ms338.8 ms0
cpoll_cppsp-raw87.5 ms
1.4%
69.3 ms1340.0 ms0
http4s89.6 ms
1.4%
42.1 ms852.9 ms0
aiohttp-pg-raw91.0 ms
1.4%
27.4 ms740.4 ms0
fintrospect95.2 ms
1.5%
46.7 ms899.0 ms0
cutelyst-thread-mysq96.0 ms
1.5%
43.7 ms442.1 ms0
silicon96.6 ms
1.5%
141.2 ms1790.0 ms53
servlet-raw96.9 ms
1.5%
22.3 ms384.9 ms0
cutelyst-thread-mysq98.2 ms
1.5%
50.6 ms522.6 ms0
wildfly-ee799.1 ms
1.5%
46.6 ms722.2 ms0
kami99.4 ms
1.5%
27.4 ms586.7 ms0
api_hour-mysql101.3 ms
1.6%
66.7 ms1040.0 ms0
actframework-hiberna103.2 ms
1.6%
15.7 ms269.8 ms0
web2py-optimized104.0 ms
1.6%
45.9 ms448.7 ms0
play1104.1 ms
1.6%
37.1 ms900.3 ms0
cppcms108.3 ms
1.7%
21.5 ms866.1 ms0
cppcms-direct108.5 ms
1.7%
14.9 ms364.6 ms0
cutelyst-nginx-mysql108.7 ms
1.7%
5.0 ms126.7 ms0
actframework-morphia109.2 ms
1.7%
24.4 ms242.8 ms0
cutelyst-pf-mysql110.7 ms
1.7%
48.4 ms386.3 ms0
actframework-ebean-p110.9 ms
1.7%
20.2 ms329.6 ms0
morepath117.1 ms
1.8%
38.1 ms400.0 ms0
wt120.4 ms
1.9%
183.6 ms2830.0 ms0
actframework-hiberna120.5 ms
1.9%
13.1 ms334.4 ms0
weppy-py3125.0 ms
1.9%
47.5 ms393.5 ms0
actframework-ebean-m126.1 ms
2.0%
17.8 ms403.3 ms0
grizzly-jersey126.6 ms
2.0%
21.0 ms340.5 ms0
undertow-jersey-hika135.8 ms
2.1%
60.7 ms570.3 ms0
ninja-standalone139.0 ms
2.2%
65.1 ms1830.0 ms0
cutelyst-pf-mysql-ep139.6 ms
2.2%
146.0 ms1830.0 ms0
tornado-pypy2144.1 ms
2.2%
76.9 ms748.1 ms0
go-interpolate147.0 ms
2.3%
22.7 ms244.1 ms0
tapestry152.9 ms
2.4%
34.0 ms537.5 ms0
weppy158.9 ms
2.5%
69.9 ms627.5 ms0
dropwizard-postgres159.7 ms
2.5%
46.3 ms424.8 ms0
weppy-nginx-uwsgi160.6 ms
2.5%
26.5 ms473.4 ms0
dropwizard-mysql163.0 ms
2.5%
50.0 ms433.3 ms0
play2-scala-anorm-li170.6 ms
2.6%
36.2 ms712.2 ms0
gin172.3 ms
2.7%
47.2 ms491.1 ms0
dropwizard-mongodb174.8 ms
2.7%
101.2 ms491.3 ms15,783
goji178.5 ms
2.8%
43.5 ms390.6 ms0
vertx-web-mongodb180.0 ms
2.8%
85.7 ms682.2 ms12
api_hour180.3 ms
2.8%
184.2 ms1970.0 ms0
revel-raw180.4 ms
2.8%
51.8 ms514.6 ms0
web2py181.6 ms
2.8%
97.8 ms852.5 ms0
undertow-mongodb181.8 ms
2.8%
155.0 ms798.2 ms10,091
falcore181.9 ms
2.8%
43.7 ms385.9 ms0
play2-scala-anorm-li182.6 ms
2.8%
27.5 ms504.9 ms0
revel-qbs183.7 ms
2.8%
52.6 ms474.6 ms0
undertow-mongodb-asy189.7 ms
2.9%
175.9 ms637.4 ms266
revel-jet189.9 ms
2.9%
53.1 ms523.8 ms0
treefrog-hybrid193.1 ms
3.0%
36.0 ms657.5 ms2
wicket193.4 ms
3.0%
76.5 ms819.6 ms0
activeweb-jackson202.6 ms
3.1%
37.8 ms606.7 ms0
flask-py3203.7 ms
3.2%
118.3 ms1110.0 ms0
aiohttp210.0 ms
3.3%
95.0 ms783.6 ms0
fasthttp210.4 ms
3.3%
52.2 ms691.8 ms0
play2-java-jpa-hikar215.4 ms
3.3%
38.8 ms597.7 ms0
activeweb217.4 ms
3.4%
47.0 ms496.1 ms0
treefrog221.7 ms
3.4%
42.7 ms706.3 ms0
cppcms-postgres222.4 ms
3.4%
200.8 ms1430.0 ms0
go222.9 ms
3.5%
35.8 ms390.1 ms0
gemini-mysql223.4 ms
3.5%
245.7 ms2330.0 ms0
play2-java-jpa-hikar223.9 ms
3.5%
24.4 ms534.0 ms0
go-postgres224.4 ms
3.5%
39.8 ms515.3 ms0
flask-nginx-uwsgi226.0 ms
3.5%
43.0 ms491.5 ms0
echo226.1 ms
3.5%
42.1 ms544.7 ms0
aah228.6 ms
3.5%
51.0 ms718.7 ms0
django-py3241.3 ms
3.7%
131.1 ms871.5 ms0
flask243.2 ms
3.8%
139.1 ms1160.0 ms0
uvicorn245.3 ms
3.8%
143.9 ms1200.0 ms0
play2-scala-slick248.0 ms
3.8%
541.7 ms7890.0 ms247
play2-java-jooq-hika250.4 ms
3.9%
26.7 ms550.0 ms0
treefrog-postgres266.6 ms
4.1%
71.9 ms1090.0 ms0
bottle-nginx-uwsgi275.7 ms
4.3%
47.5 ms746.9 ms0
django-postgresql286.1 ms
4.4%
146.0 ms867.3 ms0
pyramid286.6 ms
4.4%
159.1 ms1240.0 ms0
wheezyweb287.0 ms
4.4%
167.7 ms1220.0 ms0
bottle-py3287.7 ms
4.5%
169.8 ms1470.0 ms0
turbogears294.4 ms
4.6%
135.9 ms973.7 ms0
django296.4 ms
4.6%
141.3 ms1070.0 ms0
bottle301.6 ms
4.7%
157.2 ms1400.0 ms0
restexpress308.0 ms
4.8%
117.6 ms1370.0 ms0
tornado-py3314.1 ms
4.9%
160.2 ms1330.0 ms0
cherrypy-py3314.6 ms
4.9%
475.2 ms3750.0 ms366
dropwizard-jdbi-post323.3 ms
5.0%
396.7 ms2310.0 ms102
pyramid-py2324.0 ms
5.0%
173.9 ms1300.0 ms0
spring324.1 ms
5.0%
49.4 ms527.8 ms0
tornado331.4 ms
5.1%
91.3 ms911.3 ms0
restexpress-mysql-ra332.4 ms
5.1%
594.4 ms3660.0 ms76,363
cherrypy379.7 ms
5.9%
554.6 ms4050.0 ms218
flask-pypy2-mysql-ra379.9 ms
5.9%
416.4 ms6210.0 ms0
play2-java-ebean-hik388.9 ms
6.0%
64.8 ms1120.0 ms0
play2-java-ebean-hik392.1 ms
6.1%
47.4 ms534.7 ms0
gemini-postgres431.6 ms
6.7%
878.0 ms7990.0 ms0
play2-scala-reactive433.1 ms
6.7%
296.3 ms1690.0 ms0
blade441.5 ms
6.8%
141.7 ms1180.0 ms0
weppy-pypy2452.0 ms
7.0%
614.4 ms7560.0 ms0
beego-orm-mysql463.2 ms
7.2%
100.2 ms1230.0 ms0
play2-scala-reactive492.6 ms
7.6%
349.7 ms2170.0 ms0
go-mongo630.6 ms
9.8%
3.5 ms652.8 ms0
play2-scala-slick-ne920.7 ms
14.3%
1250.0 ms7810.0 ms0
play2-java-jooq-hika962.6 ms
14.9%
1420.0 ms6140.0 ms0
undertow-jersey-c3p01140.0 ms
17.6%
1070.0 ms7220.0 ms0
tornado-postgresql-r1290.0 ms
20.0%
1320.0 ms5190.0 ms911
ffead-cpp-apache-pos1300.0 ms
20.1%
250.9 ms1510.0 ms0
ffead-cpp-nginx-post1350.0 ms
20.9%
208.0 ms1550.0 ms0
spark1480.0 ms
22.9%
1460.0 ms7510.0 ms0
flask-pypy21540.0 ms
23.8%
1210.0 ms7990.0 ms0
bottle-pypy21580.0 ms
24.5%
1100.0 ms8000.0 ms0
klein5410.0 ms
83.7%
1200.0 ms7060.0 ms443
webware6460.0 ms
100.0%
678.1 ms7990.0 ms807
Framework overhead of 20-query responses, Dell servers at ServerCentral
Framework bestvsPlatform bestRatio (higher is better)
undertow-jersey-hika3,652vsundertow-jersey-c3p0454
804.4%
treefrog-mongodb6,688vstreefrog2,216
301.8%
fasthttp-postgresql6,091vsgo2,279
267.3%
go-prefork6,072vsgo2,279
266.4%
tornado-pypy23,438vstornado1,501
229.0%
kami4,938vsgo2,279
216.7%
wt-postgres9,349vswt5,433
172.1%
go-interpolate3,467vsgo2,279
152.1%
gin2,870vsgo2,279
125.9%
goji2,850vsgo2,279
125.1%
servlet-postgres-raw6,341vsservlet-raw5,100
124.3%
falcore2,797vsgo2,279
122.7%
revel-raw2,716vsgo2,279
119.2%
revel-qbs2,603vsgo2,279
114.2%
revel-jet2,580vsgo2,279
113.2%
treefrog-hybrid2,497vstreefrog2,216
112.7%
tornado-py31,589vstornado1,501
105.9%
fasthttp2,350vsgo2,279
103.1%
go-postgres2,265vsgo2,279
99.4%
echo2,247vsgo2,279
98.6%
aah2,221vsgo2,279
97.5%
dropwizard-mysql2,970vsservlet-raw5,100
58.2%
dropwizard-postgres3,092vsservlet-postgres-raw6,341
48.8%
play2-scala-anorm-li2,826vsakka-http5,897
47.9%
beego-orm-mysql1,087vsgo2,279
47.7%
play2-java-jpa-hikar2,311vsakka-http5,897
39.2%
go-mongo785vsgo2,279
34.4%
dropwizard-jdbi-post2,108vsservlet-postgres-raw6,341
33.2%
tornado-postgresql-r376vstornado1,501
25.0%
play2-java-ebean-hik1,272vsakka-http5,897
21.6%
play2-java-jooq-hika1,241vsakka-http5,897
21.0%
play2-scala-reactive967vsakka-http5,897
16.4%
undertow-jersey-c3p0454vsundertow-jersey-hika3,652
12.4%
play2-scala-slick689vsakka-http5,897
11.7%
20-queries (bar)
Data table
Latency
Framework overhead
20-queries (bar)
Data table
Latency
Framework overhead

Requirements summary

In this test, each request is processed by fetching multiple rows from a simple database table and serializing these rows as a JSON response. The test is run multiple times: testing 1, 5, 10, 15, and 20 queries per request. All tests are run at 256 concurrency.

Example response for 10 queries:

HTTP/1.1 200 OK Content-Length: 315 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT [{"id":4174,"randomNumber":331},{"id":51,"randomNumber":6544},{"id":4462,"randomNumber":952},{"id":2221,"randomNumber":532},{"id":9276,"randomNumber":3097},{"id":3056,"randomNumber":7293},{"id":6964,"randomNumber":620},{"id":675,"randomNumber":6601},{"id":8414,"randomNumber":6569},{"id":2753,"randomNumber":4065}]

For a more detailed description of the requirements, see the Source Code and Requirements section.

Responses per second at 20 updates per request, Dell servers at ServerCentral(138 tests)
FrameworkPerformance (higher is better)ClsLngPltFEAosIAErrors
ulib-postgres_fit4,886
100.0%(95.1%)
PltC++NonULiLinRea0
vertx-web-postgres4,002
81.9%(77.9%)
McrJavvtxNonLinRea0
ulib-postgres3,729
76.3%(72.6%)
PltC++NonULiLinRea0
vertx-postgres3,478
71.2%(67.7%)
PltJavVerNonLinRea0
undertow-postgresql3,352
68.6%(65.3%)
PltJavUtwNonLinRea0
ulib-mongodb3,313
67.8%(64.5%)
PltC++NonULiLinRea0
jawn3,252
66.6%(63.3%)
FulJavSvtUtwLinRea0
dropwizard-postgres3,243
66.4%(63.1%)
FulJavJAXJtyLinRea0
tornado-pypy23,185
65.2%(62.0%)
PltPyNonTorLinRea0
cutelyst-nginx-postg3,073
62.9%(59.8%)
FulC++QtngxLinRea0
cutelyst-pf-postgres3,057
62.6%(59.5%)
FulC++QtNonLinRea0
wt-postgres2,975
60.9%(57.9%)
FulC++NonNonLinRea6
undertow-mongodb-asy2,873
58.8%(55.9%)
PltJavUtwNonLinRea18
dropwizard-mysql2,820
57.7%(54.9%)
FulJavJAXJtyLinRea0
cutelyst-thread-post2,801
57.3%(54.5%)
FulC++QtNonLinRea0
cutelyst-thread-post2,794
57.2%(54.4%)
FulC++QtNonLinRea0
fasthttp-postgresql2,702
55.3%(52.6%)
PltGoNonNonLinRea0
treefrog-mongodb2,608
53.4%(50.8%)
FulC++NonNonLinRea0
ulib-mysql2,468
50.5%(48.1%)
PltC++NonULiLinRea0
undertow-mysql2,453
50.2%(47.8%)
PltJavUtwNonLinRea0
undertow-mongodb2,404
49.2%(46.8%)
PltJavUtwNonLinRea10,745
actframework-morphia2,285
46.8%(44.5%)
FulJavUtwNonLinRea0
jooby2,264
46.3%(44.1%)
FulJavNtyNonLinRea0
revenj-jvm2,255
46.2%(43.9%)
FulJavSvtResLinRea0
fintrospect2,241
45.9%(43.6%)
McrScaNtyNonLinRea0
minijax2,182
44.7%(42.5%)
FulJavJAXJtyLinRea0
akka-http2,117
43.3%(41.2%)
McrScaAkkNonLinRea0
light-4j-postgresql2,058
42.1%(40.1%)
PltJavLigNonLinRea0
cutelyst-thread-mysq2,054
42.0%(40.0%)
FulC++QtNonLinRea0
wt2,040
41.8%(39.7%)
FulC++NonNonLinRea45
silicon2,038
41.7%(39.7%)
McrC++NonmicLinRea0
cpoll_cppsp-raw2,034
41.6%(39.6%)
PltC++NonNonLinRea0
cutelyst-thread-mysq2,016
41.3%(39.3%)
FulC++QtNonLinRea0
go-interpolate1,913
39.2%(37.2%)
PltGoNonNonLinRea0
servlet-raw1,850
37.9%(36.0%)
PltJavSvtResLinRea0
flask-mysql-raw1,846
37.8%(35.9%)
McrPyMeiNonLinRea0
fasthttp1,840
37.7%(35.8%)
PltGoNonNonLinRea0
cutelyst-nginx-mysql1,831
37.5%(35.7%)
FulC++QtngxLinRea0
cutelyst-pf-mysql1,828
37.4%(35.6%)
FulC++QtNonLinRea0
cutelyst-pf-mysql-ep1,823
37.3%(35.5%)
FulC++QtNonLinRea0
bottle-mysql-raw1,804
36.9%(35.1%)
McrPyMeiNonLinRea0
morepath1,789
36.6%(34.8%)
McrPyMeiGunLinRea0
actframework-ebean-m1,772
36.3%(34.5%)
FulJavUtwNonLinRea0
dropwizard-jdbi-post1,749
35.8%(34.1%)
FulJavJAXJtyLinRea0
actframework-hiberna1,723
35.3%(33.5%)
FulJavUtwNonLinRea0
actframework-ebean-p1,694
34.7%(33.0%)
FulJavUtwNonLinRea0
actframework-hiberna1,680
34.4%(32.7%)
FulJavUtwNonLinRea0
falcore1,650
33.8%(32.1%)
McrGoNonNonLinRea0
gin1,649
33.7%(32.1%)
McrGoNonNonLinRea0
wildfly-ee71,633
33.4%(31.8%)
FulJavSvtWilLinRea0
cppcms-postgres1,589
32.5%(30.9%)
PltC++NonNonLinRea0
weppy-py31,483
30.4%(28.9%)
FulPyMeiNonLinRea0
go-prefork1,481
30.3%(28.8%)
PltGoNonNonLinRea0
play2-scala-anorm-li1,480
30.3%(28.8%)
FulScaAkkNonLinRea0
play2-scala-anorm-li1,474
30.2%(28.7%)
FulScaNtyNonLinRea0
servlet-postgres-raw1,471
30.1%(28.6%)
PltJavSvtResLinRea0
revel-raw1,467
30.0%(28.6%)
FulGoNonNonLinRea0
vertx-web-mongodb1,433
29.3%(27.9%)
McrJavvtxNonLinRea23
go1,422
29.1%(27.7%)
PltGoNonNonLinRea0
go-postgres1,412
28.9%(27.5%)
PltGoNonNonLinRea0
pyramid1,407
28.8%(27.4%)
FulPyNonMeiLinRea0
aah1,406
28.8%(27.4%)
FulGoNonNonLinRea0
tornado1,398
28.6%(27.2%)
PltPyNonTorLinRea0
goji1,395
28.6%(27.2%)
McrGoNonNonLinRea0
echo1,381
28.3%(26.9%)
McrGoNonNonLinRea0
weppy1,352
27.7%(26.3%)
FulPyMeiNonLinRea0
revel-jet1,272
26.0%(24.8%)
FulGoNonNonLinRea0
flask-pypy2-mysql-ra1,270
26.0%(24.7%)
McrPyTorNonLinRea0
play2-java-jooq-hika1,214
24.8%(23.6%)
FulJavNtyNonLinRea0
pyramid-py21,209
24.7%(23.5%)
FulPyNonMeiLinRea0
kami1,185
24.3%(23.1%)
McrGoNonNonLinRea0
weppy-nginx-uwsgi1,181
24.2%(23.0%)
FulPyuWSngxLinRea0
play2-java-jooq-hika1,171
24.0%(22.8%)
FulJavAkkNonLinRea0
wicket1,165
23.8%(22.7%)
FulJavSvtResLinRea0
flask-py31,155
23.6%(22.5%)
McrPyMeiNonLinRea0
activeweb-jackson1,119
22.9%(21.8%)
FulJavSvtNonLinRea0
cppcms1,115
22.8%(21.7%)
PltC++NonNonLinRea0
play2-java-jpa-hikar1,110
22.7%(21.6%)
FulJavAkkNonLinRea13
cppcms-direct1,109
22.7%(21.6%)
PltC++NonNonLinRea0
gemini-postgres1,102
22.6%(21.5%)
FulJavSvtResLinRea0
play2-java-jpa-hikar1,087
22.2%(21.2%)
FulJavNtyNonLinRea11
flask1,077
22.0%(21.0%)
McrPyMeiNonLinRea0
api_hour1,041
21.3%(20.3%)
McrPyasyGunLinRea0
activeweb1,010
20.7%(19.7%)
FulJavSvtNonLinRea0
revel-qbs1,003
20.5%(19.5%)
FulGoNonNonLinRea0
aiohttp-pg-raw974
19.9%(19.0%)
McrPyasyGunLinRea0
play2-java-ebean-hik966
19.8%(18.8%)
FulJavNtyNonLinRea7
play2-java-ebean-hik956
19.6%(18.6%)
FulJavAkkNonLinRea10
turbogears950
19.4%(18.5%)
McrPyNonMeiLinRea0
tornado-py3940
19.2%(18.3%)
PltPyNonTorLinRea0
bottle-py3935
19.1%(18.2%)
McrPyMeiNonLinRea0
treefrog-hybrid927
19.0%(18.0%)
FulC++NonNonLinRea1
bottle919
18.8%(17.9%)
McrPyMeiNonLinRea0
aiohttp890
18.2%(17.3%)
McrPyasyGunLinRea0
flask-nginx-uwsgi870
17.8%(16.9%)
McrPyNonngxLinRea0
dropwizard-mongodb868
17.8%(16.9%)
FulJavJAXJtyLinRea17,055
django-py3857
17.5%(16.7%)
FulPyNonMeiLinRea0
vertx-web-susom-post828
16.9%(16.1%)
McrJavvtxNonLinRea0
gemini-mysql819
16.8%(15.9%)
FulJavSvtResLinRea0
beego-orm-mysql801
16.4%(15.6%)
FulGoNonNonLinRea0
wheezyweb777
15.9%(15.1%)
McrPyNonMeiLinRea0
uvicorn760
15.6%(14.8%)
PltPyNonNonLinRea0
bottle-nginx-uwsgi737
15.1%(14.3%)
McrPyuWSngxLinRea0
django-postgresql710
14.5%(13.8%)
FulPyNonMeiLinRea0
treefrog703
14.4%(13.7%)
FulC++NonNonLinRea0
django683
14.0%(13.3%)
FulPyNonMeiLinRea0
spring659
13.5%(12.8%)
FulJavUtwNonLinRea0
ninja-standalone653
13.4%(12.7%)
FulJavJtyNonLinRea0
treefrog-postgres568
11.6%(11.1%)
FulC++NonNonLinRea0
web2py-optimized513
10.5%(10.0%)
FulPyMeiNonLinRea0
play2-scala-slick360
7.4%(7.0%)
FulScaAkkNonLinRea99
play2-scala-slick-ne354
7.2%(6.9%)
FulScaNtyNonLinRea0
go-mongo294
6.0%(5.7%)
PltGoNonNonLinRea0
ffead-cpp-apache-pos207
4.2%(4.0%)
FulC++ffeApcLinRea0
ffead-cpp-nginx-post207
4.2%(4.0%)
FulC++ffengxLinRea0
http4s116
2.4%(2.3%)
McrScaNIOblaLinRea35
web2py107
2.2%(2.1%)
FulPyMeiNonLinRea1,055
bottle-pypy295
1.9%(1.8%)
McrPyTorNonLinRea0
flask-pypy290
1.8%(1.8%)
McrPyNonTorLinRea0
spark85
1.7%(1.7%)
McrJavSvtResLinRea0
play2-scala-reactive56
1.1%(1.1%)
FulScaAkkNonLinRea0
play2-scala-reactive48
1.0%(0.9%)
FulScaNtyNonLinRea0
cherrypy-py315
0.3%(0.3%)
McrPyNonNonLinRea192
weppy-pypy211
0.2%(0.2%)
FulPyTorNonLinRea142
klein8
0.2%(0.2%)
McrPyNonTwiLinRea40
cherrypy8
0.2%(0.2%)
McrPyNonNonLinRea330
api_hour-mysql0
0.0%(0.0%)
McrPyasyGunLinRea188
ffead-cpp0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-apache-mon0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-apache-mys0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-mysql0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-nginx-mong0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-nginx-mysq0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-postgresql0
0.0%(0.0%)
FulC++ffeffeLinRea0
webware0
0.0%(0.0%)
McrPyNonNonLinRea795
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
Responses per second, multiple updates, Dell servers at ServerCentral
Framework15101520ClsLngPltFEAosIAErrors
aah14,2624,6962,6361,8381,406FulGoNonNonLinRea0
actframework-ebean-m35,2987,0993,5652,3661,772FulJavUtwNonLinRea0
actframework-ebean-p33,1266,7353,3872,2661,694FulJavUtwNonLinRea0
actframework-hiberna35,6647,1233,4852,3341,723FulJavUtwNonLinRea0
actframework-hiberna32,4306,6893,3892,2501,680FulJavUtwNonLinRea0
actframework-morphia45,9369,1124,5533,0612,285FulJavUtwNonLinRea0
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
activeweb12,9843,7631,9751,3331,010FulJavSvtNonLinRea0
activeweb-jackson23,1624,6622,3281,5521,119FulJavSvtNonLinRea0
aiohttp17,3283,6821,7991,201890McrPyasyGunLinRea0
aiohttp-pg-raw10,8223,2601,7741,258974McrPyasyGunLinRea0
akka-http42,7108,3744,2252,8282,117McrScaAkkNonLinRea0
api_hour16,5763,5741,8571,2891,041McrPyasyGunLinRea0
api_hour-mysql9,2132001100McrPyasyGunLinRea188
beego-orm-mysql17,5703,4911,6891,095801FulGoNonNonLinRea0
bottle11,5853,1531,6111,232919McrPyMeiNonLinRea0
bottle-mysql-raw37,2577,3663,6872,4411,804McrPyMeiNonLinRea0
bottle-nginx-uwsgi5,1243,4711,7701,093737McrPyuWSngxLinRea0
bottle-py312,3503,4061,8721,351935McrPyMeiNonLinRea0
bottle-pypy21,31036519413795McrPyTorNonLinRea0
cpoll_cppsp-raw40,6637,9424,0352,6442,034PltC++NonNonLinRea0
cutelyst-nginx-mysql34,7727,3593,7032,4731,831FulC++QtngxLinRea0
cutelyst-nginx-postg57,13312,1646,1204,0903,073FulC++QtngxLinRea0
cutelyst-pf-mysql35,4847,3893,7082,4031,828FulC++QtNonLinRea0
cutelyst-pf-mysql-ep31,8457,2233,6962,4331,823FulC++QtNonLinRea0
cutelyst-pf-postgres58,32812,1566,1034,0803,057FulC++QtNonLinRea0
cutelyst-thread-mysq41,6408,2684,1682,7202,054FulC++QtNonLinRea0
cutelyst-thread-mysq41,3358,1974,1562,7372,016FulC++QtNonLinRea0
cutelyst-thread-post58,17611,3455,6393,7522,801FulC++QtNonLinRea0
cutelyst-thread-post58,14111,3295,5913,7422,794FulC++QtNonLinRea0
django9,7642,6111,359920683FulPyNonMeiLinRea0
django-postgresql10,8732,7591,449979710FulPyNonMeiLinRea0
django-py313,4863,3631,7241,169857FulPyNonMeiLinRea0
dropwizard-jdbi-post31,6696,7813,4922,3401,749FulJavJAXJtyLinRea0
dropwizard-mongodb22,9214,3952,1891,170868FulJavJAXJtyLinRea17,055
dropwizard-mysql46,29213,4715,9703,9352,820FulJavJAXJtyLinRea0
dropwizard-postgres47,24314,3476,7134,3603,243FulJavJAXJtyLinRea0
falcore28,3967,3473,6272,3791,650McrGoNonNonLinRea0
ffead-cpp00000FulC++ffeffeLinRea0
ffead-cpp-apache-mon00000FulC++ffeApcLinRea0
ffead-cpp-apache-mys00000FulC++ffeApcLinRea0
ffead-cpp-apache-pos4,029833418280207FulC++ffeApcLinRea0
ffead-cpp-mysql00000FulC++ffeffeLinRea0
ffead-cpp-nginx-mong00000FulC++ffengxLinRea0
ffead-cpp-nginx-mysq00000FulC++ffengxLinRea0
ffead-cpp-nginx-post4,005832417277207FulC++ffengxLinRea0
ffead-cpp-postgresql00000FulC++ffeffeLinRea0
fintrospect42,8879,2624,5553,0612,241McrScaNtyNonLinRea0
flask10,1963,3871,8951,4651,077McrPyMeiNonLinRea0
flask-mysql-raw25,5587,3893,7062,4941,846McrPyMeiNonLinRea0
flask-nginx-uwsgi5,7243,4821,9411,274870McrPyNonngxLinRea0
flask-py311,7313,9042,3091,6891,155McrPyMeiNonLinRea0
flask-pypy21,10032418613090McrPyNonTorLinRea0
flask-pypy2-mysql-ra3,2272,0831,7701,5071,270McrPyTorNonLinRea0
gemini-mysql17,1553,1911,5871,121819FulJavSvtResLinRea0
gemini-postgres19,6414,1351,9431,3811,102FulJavSvtResLinRea0
gin30,2847,6733,8502,4161,649McrGoNonNonLinRea0
goji23,6276,6723,0661,9581,395McrGoNonNonLinRea0
http4s41,17011,3224,259671116McrScaNIOblaLinRea35
jawn61,62212,8906,4584,3183,252FulJavSvtUtwLinRea0
jooby43,9129,0214,5393,0222,264FulJavNtyNonLinRea0
kami29,2535,3922,5891,6521,185McrGoNonNonLinRea0
klein2085934128McrPyNonTwiLinRea40
minijax21,4577,0374,0262,8382,182FulJavJAXJtyLinRea0
morepath21,2546,5563,5212,3891,789McrPyMeiGunLinRea0
ninja-standalone7,4902,5631,316872653FulJavJtyNonLinRea0
cherrypy1676735208McrPyNonNonLinRea330
cherrypy-py322179392215McrPyNonNonLinRea192
cppcms13,4774,4872,2411,4841,115PltC++NonNonLinRea0
cppcms-direct10,1124,5212,2491,4831,109PltC++NonNonLinRea0
cppcms-postgres14,1906,3463,1952,1301,589PltC++NonNonLinRea0
echo15,7174,8172,7051,8511,381McrGoNonNonLinRea0
fasthttp11,9905,3693,2892,3921,840PltGoNonNonLinRea0
fasthttp-postgresql38,93311,5996,5003,8802,702PltGoNonNonLinRea0
go15,0484,9892,7331,9081,422PltGoNonNonLinRea0
go-interpolate16,4646,3613,7272,5601,913PltGoNonNonLinRea0
go-mongo6,4401,271627409294PltGoNonNonLinRea0
go-postgres16,6835,0372,6931,8531,412PltGoNonNonLinRea0
go-prefork29,4255,7202,9411,9081,481PltGoNonNonLinRea0
light-4j-postgresql41,3618,2784,1292,7692,058PltJavLigNonLinRea0
revel-jet23,0704,6762,4221,6411,272FulGoNonNonLinRea0
revel-qbs17,9443,9001,9911,3431,003FulGoNonNonLinRea0
revel-raw27,2355,9392,9581,9911,467FulGoNonNonLinRea0
servlet-postgres-raw28,7885,8622,9581,9741,471PltJavSvtResLinRea0
servlet-raw36,1117,6693,7052,4941,850PltJavSvtResLinRea0
tornado9,7274,3122,5301,8241,398PltPyNonTorLinRea0
tornado-py310,7453,3331,8341,240940PltPyNonTorLinRea0
tornado-pypy228,03911,4646,2084,2213,185PltPyNonTorLinRea0
undertow-mongodb29,4469,3414,8183,1652,404PltJavUtwNonLinRea10,745
undertow-mongodb-asy25,4749,1914,9703,6142,873PltJavUtwNonLinRea18
undertow-mysql48,7659,9114,8733,2742,453PltJavUtwNonLinRea0
undertow-postgresql66,88313,2906,6944,4513,352PltJavUtwNonLinRea0
vertx-postgres33,18210,7036,2434,4753,478PltJavVerNonLinRea0
wildfly-ee723,5805,8452,5302,2331,633FulJavSvtWilLinRea0
play2-java-ebean-hik11,2233,3741,8721,278956FulJavAkkNonLinRea10
play2-java-ebean-hik11,0123,4041,8521,268966FulJavNtyNonLinRea7
play2-java-jooq-hika14,3513,8492,3731,3651,171FulJavAkkNonLinRea0
play2-java-jooq-hika14,6234,1622,4251,3761,214FulJavNtyNonLinRea0
play2-java-jpa-hikar11,1873,6152,0271,4521,110FulJavAkkNonLinRea13
play2-java-jpa-hikar11,5523,7702,1371,4201,087FulJavNtyNonLinRea11
play2-scala-anorm-li26,9545,9362,9221,9881,480FulScaAkkNonLinRea0
play2-scala-anorm-li26,3215,6752,8501,9541,474FulScaNtyNonLinRea0
play2-scala-reactive1,6903181367356FulScaAkkNonLinRea0
play2-scala-reactive1,7193091477648FulScaNtyNonLinRea0
play2-scala-slick8,0161,567765496360FulScaAkkNonLinRea99
play2-scala-slick-ne7,6731,522742480354FulScaNtyNonLinRea0
pyramid12,3264,5162,5091,7981,407FulPyNonMeiLinRea0
pyramid-py210,5273,9222,2361,5571,209FulPyNonMeiLinRea0
revenj-jvm23,9998,8904,6943,0672,255FulJavSvtResLinRea0
silicon25,2607,4213,9072,6372,038McrC++NonmicLinRea0
spark1,56047225912985McrJavSvtResLinRea0
spring13,0322,5961,301857659FulJavUtwNonLinRea0
treefrog14,2362,8731,430936703FulC++NonNonLinRea0
treefrog-hybrid18,9163,7951,8891,239927FulC++NonNonLinRea1
treefrog-mongodb52,87710,5995,2453,4872,608FulC++NonNonLinRea0
treefrog-postgres12,2672,3811,219793568FulC++NonNonLinRea0
turbogears10,1543,0361,6261,249950McrPyNonMeiLinRea0
ulib-mongodb66,34813,2596,6534,4123,313PltC++NonULiLinRea0
ulib-mysql49,3319,8024,8723,2952,468PltC++NonULiLinRea0
ulib-postgres25,30611,1416,7404,8093,729PltC++NonULiLinRea0
ulib-postgres_fit31,32214,9069,0486,3634,886PltC++NonULiLinRea0
uvicorn9,7142,8381,4941,028760PltPyNonNonLinRea0
vertx-web-mongodb29,1885,7952,8721,9221,433McrJavvtxNonLinRea23
vertx-web-postgres46,85914,1137,6625,2214,002McrJavvtxNonLinRea0
vertx-web-susom-post36,97811,6102,987837828McrJavvtxNonLinRea0
web2py7,0033,9241,114267107FulPyMeiNonLinRea1,055
web2py-optimized13,0103,201684543513FulPyMeiNonLinRea0
webware00000McrPyNonNonLinRea795
weppy17,7724,9042,6851,9561,352FulPyMeiNonLinRea0
weppy-nginx-uwsgi4,6996,1833,0981,8341,181FulPyuWSngxLinRea0
weppy-py322,5526,2893,1752,1401,483FulPyMeiNonLinRea0
weppy-pypy22,9711,0324071911FulPyTorNonLinRea142
wheezyweb12,2693,1061,5611,086777McrPyNonMeiLinRea0
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
wicket19,1054,4742,2901,5661,165FulJavSvtResLinRea0
wt41,5128,2464,0702,7412,040FulC++NonNonLinRea45
wt-postgres55,89911,9805,9723,9812,975FulC++NonNonLinRea6
Latency of 20-update responses, Dell servers at ServerCentral
FrameworkAverage latency (lower is better)σ (SD)MaxErrors
minijax89.3 ms
1.4%
38.6 ms876.6 ms0
ulib-postgres_fit104.7 ms
1.6%
44.1 ms287.9 ms0
vertx-web-postgres125.2 ms
1.9%
35.2 ms365.5 ms0
ulib-postgres143.0 ms
2.2%
94.1 ms946.6 ms0
undertow-postgresql149.6 ms
2.3%
67.2 ms713.0 ms0
vertx-postgres150.6 ms
2.3%
82.5 ms1200.0 ms0
jawn153.2 ms
2.4%
66.7 ms745.4 ms0
ulib-mongodb154.3 ms
2.4%
75.7 ms446.1 ms0
dropwizard-postgres157.3 ms
2.4%
47.1 ms489.2 ms0
cutelyst-nginx-postg165.8 ms
2.5%
9.7 ms196.1 ms0
tornado-pypy2166.9 ms
2.6%
129.1 ms1170.0 ms0
cutelyst-pf-postgres168.5 ms
2.6%
74.5 ms661.3 ms0
undertow-mongodb-asy178.7 ms
2.7%
164.8 ms887.6 ms18
cutelyst-thread-post183.7 ms
2.8%
91.5 ms647.9 ms0
cutelyst-thread-post184.4 ms
2.8%
82.9 ms731.6 ms0
fasthttp-postgresql186.1 ms
2.9%
72.0 ms636.0 ms0
dropwizard-mysql189.8 ms
2.9%
124.2 ms1070.0 ms0
undertow-mongodb191.6 ms
2.9%
157.6 ms818.0 ms10,745
treefrog-mongodb192.0 ms
3.0%
111.8 ms1170.0 ms0
ulib-mysql206.4 ms
3.2%
91.3 ms571.3 ms0
undertow-mysql208.9 ms
3.2%
87.4 ms1010.0 ms0
fintrospect211.4 ms
3.3%
68.1 ms1010.0 ms0
actframework-morphia214.1 ms
3.3%
42.5 ms433.3 ms0
api_hour-mysql219.0 ms
3.4%
253.0 ms1130.0 ms188
jooby224.6 ms
3.5%
45.7 ms583.0 ms0
silicon233.0 ms
3.6%
132.7 ms1850.0 ms0
akka-http233.7 ms
3.6%
47.7 ms575.0 ms0
dropwizard-mongodb244.6 ms
3.8%
177.9 ms539.8 ms17,055
light-4j-postgresql246.7 ms
3.8%
31.0 ms414.2 ms0
cutelyst-thread-mysq248.6 ms
3.8%
116.1 ms835.5 ms0
cpoll_cppsp-raw251.0 ms
3.9%
69.5 ms938.2 ms0
cutelyst-thread-mysq254.1 ms
3.9%
110.1 ms907.8 ms0
go-interpolate255.3 ms
3.9%
67.7 ms710.6 ms0
servlet-raw259.3 ms
4.0%
90.0 ms769.7 ms0
flask-mysql-raw264.8 ms
4.1%
108.1 ms759.4 ms0
fasthttp266.0 ms
4.1%
85.0 ms884.6 ms0
bottle-mysql-raw274.4 ms
4.2%
87.2 ms992.5 ms0
dropwizard-jdbi-post276.2 ms
4.2%
59.5 ms552.3 ms0
cutelyst-nginx-mysql276.9 ms
4.3%
33.8 ms404.0 ms0
cutelyst-pf-mysql279.2 ms
4.3%
123.3 ms1060.0 ms0
vertx-web-susom-post279.3 ms
4.3%
561.6 ms3090.0 ms0
actframework-ebean-m286.1 ms
4.4%
33.7 ms453.8 ms0
actframework-hiberna294.4 ms
4.5%
34.5 ms555.2 ms0
falcore295.6 ms
4.5%
92.1 ms809.7 ms0
gin295.6 ms
4.5%
91.1 ms928.9 ms0
actframework-ebean-p296.4 ms
4.6%
49.6 ms570.6 ms0
actframework-hiberna301.1 ms
4.6%
48.5 ms543.5 ms0
morepath309.2 ms
4.8%
265.6 ms3100.0 ms0
cppcms-postgres318.5 ms
4.9%
72.7 ms707.1 ms0
wt-postgres321.6 ms
4.9%
869.3 ms8000.0 ms6
go-prefork324.4 ms
5.0%
139.8 ms1040.0 ms0
cutelyst-pf-mysql-ep329.4 ms
5.1%
310.5 ms3390.0 ms0
revel-raw332.5 ms
5.1%
98.8 ms790.6 ms0
servlet-postgres-raw332.7 ms
5.1%
98.6 ms794.1 ms0
play2-scala-anorm-li333.4 ms
5.1%
55.5 ms669.4 ms0
wt337.9 ms
5.2%
914.5 ms7980.0 ms45
go-postgres339.0 ms
5.2%
84.5 ms700.1 ms0
play2-scala-anorm-li342.2 ms
5.3%
40.1 ms653.3 ms0
pyramid348.8 ms
5.4%
170.9 ms1250.0 ms0
goji349.1 ms
5.4%
96.8 ms885.2 ms0
tornado350.3 ms
5.4%
117.4 ms961.4 ms0
vertx-web-mongodb352.4 ms
5.4%
126.1 ms1050.0 ms23
go355.8 ms
5.5%
41.2 ms511.0 ms0
aah359.5 ms
5.5%
64.2 ms785.2 ms0
revenj-jvm366.0 ms
5.6%
731.4 ms7980.0 ms0
echo366.4 ms
5.6%
43.1 ms574.9 ms0
weppy-py3378.9 ms
5.8%
355.3 ms5220.0 ms0
revel-jet396.9 ms
6.1%
97.2 ms995.1 ms0
pyramid-py2398.9 ms
6.1%
183.3 ms1490.0 ms0
flask-pypy2-mysql-ra400.9 ms
6.2%
316.0 ms5800.0 ms0
weppy409.9 ms
6.3%
429.5 ms4890.0 ms0
play2-java-jooq-hika410.5 ms
6.3%
30.0 ms596.7 ms0
wicket417.0 ms
6.4%
134.7 ms1370.0 ms0
play2-java-jooq-hika424.6 ms
6.5%
60.0 ms1150.0 ms0
kami425.5 ms
6.5%
90.2 ms1070.0 ms0
weppy-nginx-uwsgi426.5 ms
6.6%
66.8 ms924.1 ms0
activeweb-jackson436.7 ms
6.7%
96.6 ms906.2 ms0
play2-java-jpa-hikar445.2 ms
6.8%
71.8 ms1150.0 ms13
cppcms452.4 ms
7.0%
55.2 ms791.9 ms0
cppcms-direct454.6 ms
7.0%
55.3 ms810.2 ms0
play2-java-jpa-hikar462.6 ms
7.1%
54.5 ms879.6 ms11
api_hour465.7 ms
7.2%
427.7 ms4040.0 ms0
flask-py3466.7 ms
7.2%
392.2 ms3280.0 ms0
cherrypy478.5 ms
7.4%
67.6 ms725.9 ms330
activeweb482.8 ms
7.4%
114.0 ms988.0 ms0
revel-qbs485.8 ms
7.5%
94.9 ms864.3 ms0
flask496.4 ms
7.6%
423.7 ms4010.0 ms0
aiohttp-pg-raw498.4 ms
7.7%
157.8 ms1300.0 ms0
play2-java-ebean-hik503.9 ms
7.8%
57.6 ms714.2 ms7
tornado-py3507.9 ms
7.8%
123.1 ms1050.0 ms0
play2-java-ebean-hik514.1 ms
7.9%
81.2 ms1280.0 ms10
treefrog-hybrid529.1 ms
8.1%
91.3 ms1100.0 ms1
turbogears539.4 ms
8.3%
437.6 ms3720.0 ms0
aiohttp540.2 ms
8.3%
166.3 ms1220.0 ms0
django-py3546.0 ms
8.4%
184.8 ms1290.0 ms0
bottle-py3546.3 ms
8.4%
489.7 ms4200.0 ms0
flask-nginx-uwsgi576.0 ms
8.9%
92.4 ms1090.0 ms0
bottle589.8 ms
9.1%
535.2 ms6030.0 ms0
beego-orm-mysql621.6 ms
9.6%
120.5 ms1310.0 ms0
cherrypy-py3624.0 ms
9.6%
829.1 ms4760.0 ms192
uvicorn633.3 ms
9.7%
200.6 ms1560.0 ms0
gemini-postgres645.0 ms
9.9%
819.1 ms6990.0 ms0
gemini-mysql649.1 ms
10.0%
530.8 ms4180.0 ms0
wheezyweb649.5 ms
10.0%
575.3 ms4680.0 ms0
bottle-nginx-uwsgi676.9 ms
10.4%
109.8 ms1390.0 ms0
django-postgresql677.1 ms
10.4%
221.9 ms1470.0 ms0
treefrog682.2 ms
10.5%
162.6 ms1930.0 ms0
django698.6 ms
10.7%
268.3 ms1750.0 ms0
ninja-standalone737.8 ms
11.4%
204.9 ms2770.0 ms0
spring757.0 ms
11.6%
99.0 ms1060.0 ms0
treefrog-postgres829.6 ms
12.8%
264.5 ms2590.0 ms0
wildfly-ee71010.0 ms
15.5%
1530.0 ms7900.0 ms0
web2py-optimized1100.0 ms
16.9%
1370.0 ms8000.0 ms0
play2-scala-slick1200.0 ms
18.5%
1080.0 ms7980.0 ms99
play2-scala-slick-ne1390.0 ms
21.4%
906.1 ms5390.0 ms0
go-mongo1520.0 ms
23.4%
149.9 ms1770.0 ms0
ffead-cpp-apache-pos2170.0 ms
33.4%
569.7 ms2610.0 ms0
ffead-cpp-nginx-post2260.0 ms
34.8%
474.6 ms2640.0 ms0
web2py2390.0 ms
36.8%
1580.0 ms7950.0 ms1,055
bottle-pypy22820.0 ms
43.4%
1970.0 ms7980.0 ms0
flask-pypy23290.0 ms
50.6%
2050.0 ms7980.0 ms0
http4s3350.0 ms
51.5%
2330.0 ms8000.0 ms35
weppy-pypy23480.0 ms
53.5%
2550.0 ms7960.0 ms142
spark4050.0 ms
62.3%
1570.0 ms7760.0 ms0
play2-scala-reactive5870.0 ms
90.3%
1970.0 ms7980.0 ms0
klein5960.0 ms
91.7%
1480.0 ms7780.0 ms40
play2-scala-reactive6350.0 ms
97.7%
1270.0 ms7990.0 ms0
webware6500.0 ms
100.0%
668.3 ms8000.0 ms795
Framework overhead of 20-update responses, Dell servers at ServerCentral
Framework bestvsPlatform bestRatio (higher is better)
treefrog-mongodb2,608vstreefrog703
371.0%
tornado-pypy23,185vstornado1,398
227.8%
dropwizard-postgres3,243vsservlet-postgres-raw1,471
220.5%
fasthttp-postgresql2,702vsgo1,422
190.0%
dropwizard-mysql2,820vsservlet-raw1,850
152.4%
wt-postgres2,975vswt2,040
145.8%
go-interpolate1,913vsgo1,422
134.5%
treefrog-hybrid927vstreefrog703
131.9%
fasthttp1,840vsgo1,422
129.4%
dropwizard-jdbi-post1,749vsservlet-postgres-raw1,471
118.9%
falcore1,650vsgo1,422
116.0%
gin1,649vsgo1,422
116.0%
go-prefork1,481vsgo1,422
104.1%
revel-raw1,467vsgo1,422
103.2%
go-postgres1,412vsgo1,422
99.3%
aah1,406vsgo1,422
98.9%
goji1,395vsgo1,422
98.1%
echo1,381vsgo1,422
97.1%
revel-jet1,272vsgo1,422
89.5%
kami1,185vsgo1,422
83.3%
servlet-postgres-raw1,471vsservlet-raw1,850
79.5%
revel-qbs1,003vsgo1,422
70.5%
play2-scala-anorm-li1,480vsakka-http2,117
69.9%
tornado-py3940vstornado1,398
67.2%
beego-orm-mysql801vsgo1,422
56.3%
play2-java-jooq-hika1,171vsakka-http2,117
55.3%
play2-java-jpa-hikar1,110vsakka-http2,117
52.4%
play2-java-ebean-hik956vsakka-http2,117
45.2%
go-mongo294vsgo1,422
20.7%
play2-scala-slick360vsakka-http2,117
17.0%
play2-scala-reactive56vsakka-http2,117
2.6%
20-updates (bar)
Data table
Latency
Framework overhead
20-updates (bar)
Data table
Latency
Framework overhead

Requirements summary

This test exercises database writes. Each request is processed by fetching multiple rows from a simple database table, converting the rows to in-memory objects, modifying one attribute of each object in memory, updating each associated row in the database individually, and then serializing the list of objects as a JSON response. The test is run multiple times: testing 1, 5, 10, 15, and 20 updates per request. Note that the number of statements per request is twice the number of updates since each update is paired with one query to fetch the object. All tests are run at 256 concurrency.

The response is analogous to the multiple-query test. Example response for 10 updates:

HTTP/1.1 200 OK Content-Length: 315 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT [{"id":4174,"randomNumber":331},{"id":51,"randomNumber":6544},{"id":4462,"randomNumber":952},{"id":2221,"randomNumber":532},{"id":9276,"randomNumber":3097},{"id":3056,"randomNumber":7293},{"id":6964,"randomNumber":620},{"id":675,"randomNumber":6601},{"id":8414,"randomNumber":6569},{"id":2753,"randomNumber":4065}]

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best fortunes responses per second, Dell servers at ServerCentral(151 tests)
FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
ulib-postgres_fit189,705
100.0%
PltC++NonULiLinRea0
ulib-postgres184,679
97.4%
PltC++NonULiLinRea0
ulib-mongodb174,785
92.1%
PltC++NonULiLinRea0
gemini-postgres173,928
91.7%
FulJavSvtResLinRea0
cutelyst-thread-post167,932
88.5%
FulC++QtNonLinRea0
light-4j-postgresql167,583
88.3%
PltJavLigNonLinRea0
cutelyst-thread-post167,095
88.1%
FulC++QtNonLinRea0
proteus-mysql162,613
85.7%
McrJavUtwNonLinRea0
vertx-postgres146,000
77.0%
PltJavVerNonLinRea0
proteus139,484
73.5%
McrJavUtwNonLinRea0
ulib-mysql136,976
72.2%
PltC++NonULiLinRea0
undertow-postgresql134,155
70.7%
PltJavUtwNonLinRea0
fasthttp-postgresql124,514
65.6%
PltGoNonNonLinRea0
rapidoid-mysql117,301
61.8%
PltJavRapNonLinRea0
treefrog-mongodb115,114
60.7%
FulC++NonNonLinRea0
cpoll_cppsp-raw112,982
59.6%
PltC++NonNonLinRea0
go-prefork112,122
59.1%
PltGoNonNonLinRea0
rapidoid-postgres111,093
58.6%
PltJavRapNonLinRea0
cutelyst-pf-postgres110,129
58.1%
FulC++QtNonLinRea0
undertow-mysql108,367
57.1%
PltJavUtwNonLinRea0
gemini-mysql108,138
57.0%
FulJavSvtResLinRea0
cutelyst-nginx-postg100,941
53.2%
FulC++QtngxLinRea0
cutelyst-thread-mysq98,720
52.0%
FulC++QtNonLinRea0
cutelyst-thread-mysq98,159
51.7%
FulC++QtNonLinRea0
undertow-mongodb94,833
50.0%
PltJavUtwNonLinRea0
revenj-jvm87,123
45.9%
FulJavSvtResLinRea0
fintrospect85,893
45.3%
McrScaNtyNonLinRea0
actframework-ebean-p84,508
44.5%
FulJavUtwNonLinRea0
actframework-hiberna77,964
41.1%
FulJavUtwNonLinRea0
cutelyst-pf-mysql76,323
40.2%
FulC++QtNonLinRea0
cutelyst-nginx-mysql73,206
38.6%
FulC++QtngxLinRea0
jooby72,687
38.3%
FulJavNtyNonLinRea0
silicon69,547
36.7%
McrC++NonmicLinRea0
fasthttp65,152
34.3%
PltGoNonNonLinRea0
actframework-ebean-m64,286
33.9%
FulJavUtwNonLinRea0
grizzly-jersey62,635
33.0%
McrJavJAXNonLinRea0
cutelyst-pf-mysql-ep60,412
31.8%
FulC++QtNonLinRea0
kami59,441
31.3%
McrGoNonNonLinRea0
treefrog-postgres56,352
29.7%
FulC++NonNonLinRea0
actframework-hiberna56,241
29.6%
FulJavUtwNonLinRea0
treefrog55,606
29.3%
FulC++NonNonLinRea0
http4s53,768
28.3%
McrScaNIOblaLinRea0
goji53,466
28.2%
McrGoNonNonLinRea0
go-interpolate51,216
27.0%
PltGoNonNonLinRea0
go-postgres50,347
26.5%
PltGoNonNonLinRea0
go49,440
26.1%
PltGoNonNonLinRea0
servlet-postgres-raw47,985
25.3%
PltJavSvtResLinRea0
servlet-raw47,862
25.2%
PltJavSvtResLinRea0
actframework-hiberna47,433
25.0%
FulJavUtwNonLinRea0
vertx-web-postgres47,214
24.9%
McrJavvtxNonLinRea0
actframework-hiberna47,067
24.8%
FulJavUtwNonLinRea0
treefrog-hybrid46,316
24.4%
FulC++NonNonLinRea8
bottle-mysql-raw45,787
24.1%
McrPyMeiNonLinRea0
undertow-mongodb-asy45,602
24.0%
PltJavUtwNonLinRea0
vertx-web-mongodb44,949
23.7%
McrJavvtxNonLinRea0
revel-raw44,742
23.6%
FulGoNonNonLinRea0
gin44,347
23.4%
McrGoNonNonLinRea0
revel-jet42,558
22.4%
FulGoNonNonLinRea0
tornado-pypy240,210
21.2%
PltPyNonTorLinRea0
vertx-web-susom-post39,877
21.0%
McrJavvtxNonLinRea0
revel-qbs38,816
20.5%
FulGoNonNonLinRea0
minijax38,574
20.3%
FulJavJAXJtyLinRea0
falcore38,055
20.1%
McrGoNonNonLinRea0
echo36,725
19.4%
McrGoNonNonLinRea0
dropwizard-mysql35,510
18.7%
FulJavJAXJtyLinRea0
dropwizard-postgres33,729
17.8%
FulJavJAXJtyLinRea0
uvicorn32,991
17.4%
PltPyNonNonLinRea0
dropwizard-jdbi-post31,744
16.7%
FulJavJAXJtyLinRea0
aiohttp31,528
16.6%
McrPyasyGunLinRea0
aiohttp-pg-raw30,971
16.3%
McrPyasyGunLinRea0
wt30,869
16.3%
FulC++NonNonLinRea0
aah30,534
16.1%
FulGoNonNonLinRea0
api_hour30,337
16.0%
McrPyasyGunLinRea0
wt-postgres30,045
15.8%
FulC++NonNonLinRea0
tapestry29,473
15.5%
FulJavSvtResLinRea0
jawn29,316
15.5%
FulJavSvtUtwLinRea0
wheezyweb27,862
14.7%
McrPyNonMeiLinRea0
play2-java-jpa-hikar26,798
14.1%
FulJavNtyNonLinRea0
flask-mysql-raw26,198
13.8%
McrPyMeiNonLinRea0
api_hour-mysql26,068
13.7%
McrPyasyGunLinRea0
play2-java-jpa-hikar24,981
13.2%
FulJavAkkNonLinRea0
actframework-morphia24,701
13.0%
FulJavUtwNonLinRea0
morepath23,011
12.1%
McrPyMeiGunLinRea0
spring22,439
11.8%
FulJavUtwNonLinRea0
play2-java-jooq-hika22,165
11.7%
FulJavNtyNonLinRea0
weppy-py321,806
11.5%
FulPyMeiNonLinRea0
actframework-morphia21,738
11.5%
FulJavUtwNonLinRea0
play2-scala-slick-ne21,378
11.3%
FulScaNtyNonLinRea0
wicket21,168
11.2%
FulJavSvtResLinRea0
pyramid-py221,138
11.1%
FulPyNonMeiLinRea0
play2-scala-reactive20,649
10.9%
FulScaNtyNonLinRea0
pyramid20,603
10.9%
FulPyNonMeiLinRea0
weppy20,183
10.6%
FulPyMeiNonLinRea0
play2-scala-slick19,990
10.5%
FulScaAkkNonLinRea0
undertow-jersey-hika19,035
10.0%
PltJavJAXNonLinRea0
play2-java-jooq-hika18,968
10.0%
FulJavAkkNonLinRea0
django-py317,814
9.4%
FulPyNonMeiLinRea0
undertow-jersey-c3p017,594
9.3%
PltJavJAXNonLinRea0
actframework-ebean-p17,301
9.1%
FulJavUtwNonLinRea0
bottle-py316,990
9.0%
McrPyMeiNonLinRea0
play2-scala-anorm-li16,898
8.9%
FulScaNtyNonLinRea0
cppcms-postgres16,308
8.6%
PltC++NonNonLinRea0
wildfly-ee715,657
8.3%
FulJavSvtWilLinRea0
bottle15,434
8.1%
McrPyMeiNonLinRea0
play2-java-ebean-hik15,376
8.1%
FulJavAkkNonLinRea0
cppcms15,091
8.0%
PltC++NonNonLinRea0
play2-scala-anorm-li14,843
7.8%
FulScaAkkNonLinRea0
play2-java-ebean-hik14,485
7.6%
FulJavNtyNonLinRea0
flask-py314,469
7.6%
McrPyMeiNonLinRea0
turbogears14,157
7.5%
McrPyNonMeiLinRea0
activeweb13,986
7.4%
FulJavSvtNonLinRea0
tornado-py313,277
7.0%
PltPyNonTorLinRea0
play2-scala-reactive12,740
6.7%
FulScaAkkNonLinRea0
django-postgresql12,673
6.7%
FulPyNonMeiLinRea0
tornado12,672
6.7%
PltPyNonTorLinRea0
flask12,580
6.6%
McrPyMeiNonLinRea0
dropwizard-mongodb12,339
6.5%
FulJavJAXJtyLinRea0
django11,643
6.1%
FulPyNonMeiLinRea0
web2py-optimized11,262
5.9%
FulPyMeiNonLinRea0
go-mongo10,864
5.7%
PltGoNonNonLinRea0
cppcms-direct10,857
5.7%
PltC++NonNonLinRea0
flask-pypy2-mysql-ra8,741
4.6%
McrPyTorNonLinRea0
ninja-standalone7,011
3.7%
FulJavJtyNonLinRea0
bottle-pypy26,912
3.6%
McrPyTorNonLinRea0
akka-http6,753
3.6%
McrScaAkkNonLinRea0
web2py6,326
3.3%
FulPyMeiNonLinRea0
flask-nginx-uwsgi5,658
3.0%
McrPyNonngxLinRea0
weppy-nginx-uwsgi5,123
2.7%
FulPyuWSngxLinRea0
bottle-nginx-uwsgi5,015
2.6%
McrPyuWSngxLinRea0
flask-pypy23,264
1.7%
McrPyNonTorLinRea0
weppy-pypy23,132
1.7%
FulPyTorNonLinRea0
ffead-cpp-apache-pos2,861
1.5%
FulC++ffeApcLinRea0
ffead-cpp-nginx-post2,847
1.5%
FulC++ffengxLinRea0
actframework-ebean-m2,365
1.2%
FulJavUtwNonLinRea0
klein266
0.1%
McrPyNonTwiLinRea0
cherrypy-py3255
0.1%
McrPyNonNonLinRea0
spark224
0.1%
McrJavSvtResLinRea0
cherrypy197
0.1%
McrPyNonNonLinRea0
ffead-cpp0
0.0%
FulC++ffeffeLinRea0
ffead-cpp-apache-mon0
0.0%
FulC++ffeApcLinRea0
ffead-cpp-apache-mys0
0.0%
FulC++ffeApcLinRea0
ffead-cpp-mysql0
0.0%
FulC++ffeffeLinRea0
ffead-cpp-nginx-mong0
0.0%
FulC++ffengxLinRea0
ffead-cpp-nginx-mysq0
0.0%
FulC++ffengxLinRea0
ffead-cpp-postgresql0
0.0%
FulC++ffeffeLinRea0
webware0
0.0%
McrPyNonNonLinRea1,154
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
actframework-eclipseDid not completeFulJavUtwNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
Best (bar chart)
Data table
Latency
Framework overhead
Best (bar chart)
Data table
Latency
Framework overhead

Requirements summary

In this test, the framework's ORM is used to fetch all rows from a database table containing an unknown number of Unix fortune cookie messages (the table has 12 rows, but the code cannot have foreknowledge of the table's size). An additional fortune cookie message is inserted into the list at runtime and then the list is sorted by the message text. Finally, the list is delivered to the client using a server-side HTML template. The message text must be considered untrusted and properly escaped and the UTF-8 fortune messages must be rendered properly.

Whitespace is optional and may comply with the framework's best practices.

Example response:

HTTP/1.1 200 OK Content-Length: 1196 Content-Type: text/html; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT <!DOCTYPE html><html><head><title>Fortunes</title></head><body><table><tr><th>id</th><th>message</th></tr><tr><td>11</td><td>&lt;script&gt;alert(&quot;This should not be displayed in a browser alert box.&quot;);&lt;/script&gt;</td></tr><tr><td>4</td><td>A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1</td></tr><tr><td>5</td><td>A computer program does what you tell it to do, not what you want it to do.</td></tr><tr><td>2</td><td>A computer scientist is someone who fixes things that aren&apos;t broken.</td></tr><tr><td>8</td><td>A list is only as strong as its weakest link. — Donald Knuth</td></tr><tr><td>0</td><td>Additional fortune added at request time.</td></tr><tr><td>3</td><td>After enough decimal places, nobody gives a damn.</td></tr><tr><td>7</td><td>Any program that runs right is obsolete.</td></tr><tr><td>10</td><td>Computers make very fast, very accurate mistakes.</td></tr><tr><td>6</td><td>Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen</td></tr><tr><td>9</td><td>Feature: A bug with seniority.</td></tr><tr><td>1</td><td>fortune: No such file or directory</td></tr><tr><td>12</td><td>フレームワークのベンチマーク</td></tr></table></body></html>

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best JSON responses per second, Dell servers at ServerCentral(138 tests)
FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
ulib-json682,383
100.0%
PltC++NonULiLinRea0
revenj-jvm669,526
98.1%
FulJavSvtResLinRea0
ulib-json_fit664,458
97.4%
PltC++NonULiLinRea0
servlet-cjs663,623
97.3%
PltJavSvtResLinRea0
rapidoid662,269
97.1%
PltJavRapNonLinRea0
servlet-afterburner662,134
97.0%
PltJavSvtResLinRea0
proteus658,410
96.5%
McrJavUtwNonLinRea0
servlet657,791
96.4%
PltJavSvtResLinRea0
cpoll_cppsp646,776
94.8%
PltC++NonNonLinRea0
rapidoid-http-fast642,181
94.1%
PltJavRapNonLinRea0
grizzly641,988
94.1%
McrJavSvtNonLinRea0
light-4j634,889
93.0%
PltJavLigNonLinRea0
s-server633,811
92.9%
PltScas-sNonLinRea0
japronto629,566
92.3%
McrPyNonNonLinRea0
blaze617,537
90.5%
McrScablaNonLinRea0
cutelyst-thread611,406
89.6%
FulC++QtNonLinRea0
go-prefork603,592
88.5%
PltGoNonNonLinRea0
undertow595,431
87.3%
PltJavUtwNonLinRea0
netty590,560
86.5%
PltJavNtyNonLinRea0
apistar575,507
84.3%
McrPyNonMeiLinRea0
vertx-web570,243
83.6%
McrJavvtxNonLinRea0
cutelyst-thread-epol559,997
82.1%
FulC++QtNonLinRea0
vertx550,381
80.7%
PltJavVerNonLinRea0
finatra523,712
76.7%
McrScaNtyNonLinRea0
fintrospect504,706
74.0%
McrScaNtyNonLinRea0
gemini498,666
73.1%
FulJavSvtResLinRea0
colossus497,039
72.8%
McrScaAkkNonLinRea0
falcon477,082
69.9%
McrPyNonMeiLinRea0
uvicorn475,636
69.7%
PltPyNonNonLinRea0
cutelyst-thread-epol461,957
67.7%
FulC++QtNonLinRea0
falcon-py3424,771
62.2%
McrPyNonMeiLinRea0
finagle420,556
61.6%
McrScaNtyNonLinRea0
finch396,796
58.1%
McrScaNtyNonLinRea0
actframework389,125
57.0%
FulJavUtwNonLinRea0
wheezyweb387,714
56.8%
McrPyNonMeiLinRea0
bottle364,428
53.4%
McrPyMeiNonLinRea0
jlhttp360,176
52.8%
PltJavJLHNonLinRea0
fasthttp357,675
52.4%
PltGoNonNonLinRea0
blade327,933
48.1%
FulJavNonNonLinRea0
kami309,291
45.3%
McrGoNonNonLinRea0
treefrog-mongodb306,788
45.0%
FulC++NonNonLinRea0
treefrog304,265
44.6%
FulC++NonNonLinRea0
bottle-py3298,316
43.7%
McrPyMeiNonLinRea0
go296,356
43.4%
PltGoNonNonLinRea0
treefrog-postgres293,190
43.0%
FulC++NonNonLinRea0
grizzly-jersey286,502
42.0%
McrJavJAXNonLinRea0
wicket283,711
41.6%
FulJavSvtResLinRea0
gin282,655
41.4%
McrGoNonNonLinRea0
beego271,747
39.8%
FulGoNonNonLinRea0
weppy249,253
36.5%
FulPyMeiNonLinRea0
falcore229,102
33.6%
McrGoNonNonLinRea0
spark225,691
33.1%
McrJavSvtResLinRea0
sanic219,254
32.1%
McrPyNonNonLinRea0
morepath219,172
32.1%
McrPyMeiGunLinRea0
weppy-py3218,228
32.0%
FulPyMeiNonLinRea0
play2-scala-scala-ne208,400
30.5%
FulScaNtyNonLinRea0
falcon-pypy2199,972
29.3%
McrPyNonTorLinRea0
tornado-pypy2196,384
28.8%
PltPyNonTorLinRea0
aiohttp182,406
26.7%
McrPyasyGunLinRea0
jetty-servlet182,147
26.7%
PltJavJtyNonLinRea0
jetty177,118
26.0%
PltJavJtyNonLinRea0
play2-java-netty176,509
25.9%
FulJavNtyNonLinRea0
pyramid-py2175,201
25.7%
FulPyNonMeiLinRea0
jawn162,837
23.9%
FulJavSvtUtwLinRea0
activeweb-jackson161,550
23.7%
FulJavSvtNonLinRea0
pyramid157,687
23.1%
FulPyNonMeiLinRea0
silicon156,779
23.0%
McrC++NonmicLinRea0
bayou152,958
22.4%
PltJavBayNonLinRea0
spray148,368
21.7%
McrScaAkkNonLinRea0
tapestry146,931
21.5%
FulJavSvtResLinRea0
undertow-jersey-hika146,497
21.5%
PltJavJAXNonLinRea0
django-py3146,354
21.4%
FulPyNonMeiLinRea0
undertow-jersey-c3p0145,981
21.4%
PltJavJAXNonLinRea0
echo139,626
20.5%
McrGoNonNonLinRea0
webgo139,313
20.4%
McrGoNonNonLinRea0
restexpress115,937
17.0%
McrJavNtyNonLinRea0
revel112,802
16.5%
FulGoNonNonLinRea0
curacao109,172
16.0%
McrJavSvtNonLinRea5
akka-http102,259
15.0%
McrScaAkkNonLinRea0
jooby101,330
14.8%
FulJavNtyNonLinRea0
api_hour100,836
14.8%
McrPyasyGunLinRea0
cutelyst-nginx98,740
14.5%
FulC++QtngxLinRea0
comsat-webactors-und97,801
14.3%
PltJavComNonLinRea0
flask-py393,918
13.8%
McrPyMeiNonLinRea0
cutelyst-pf-epoll93,555
13.7%
FulC++QtNonLinRea0
cutelyst93,025
13.6%
FulC++QtNonLinRea0
play2-scala93,002
13.6%
FulScaAkkNonLinRea0
turbogears86,535
12.7%
McrPyNonMeiLinRea0
flask83,230
12.2%
McrPyMeiNonLinRea0
http4s82,652
12.1%
McrScaNIOblaLinRea0
goji82,014
12.0%
McrGoNonNonLinRea0
django77,278
11.3%
FulPyNonMeiLinRea0
comsat-webactors74,246
10.9%
PltJavComNonLinRea0
wildfly-ee770,103
10.3%
FulJavSvtWilLinRea0
treefrog-hybrid69,681
10.2%
FulC++NonNonLinRea18
weppy-pypy262,790
9.2%
FulPyTorNonLinRea0
wt59,385
8.7%
FulC++NonNonLinRea0
tornado59,293
8.7%
PltPyNonTorLinRea0
play258,043
8.5%
FulJavAkkNonLinRea0
activeweb57,299
8.4%
FulJavSvtNonLinRea0
comsat-servlet55,583
8.1%
PltJavSvtNonLinRea0
play155,266
8.1%
FulJavNtyNonLinRea0
scruffy54,186
7.9%
McrScaUtwNonLinRea0
minijax53,907
7.9%
FulJavJAXJtyLinRea0
tornado-py353,537
7.8%
PltPyNonTorLinRea0
web2py-optimized50,933
7.5%
FulPyMeiNonLinRea0
dropwizard44,607
6.5%
FulJavJAXJtyLinRea0
bottle-pypy243,726
6.4%
McrPyTorNonLinRea0
aah37,377
5.5%
FulGoNonNonLinRea0
comsat-servlet-under30,636
4.5%
PltJavSvtNonLinRea0
spring26,114
3.8%
FulJavUtwNonLinRea0
web2py20,848
3.1%
FulPyMeiNonLinRea0
ffead-cpp-apache-pos17,053
2.5%
FulC++ffeApcLinRea0
ffead-cpp-nginx-post16,582
2.4%
FulC++ffengxLinRea0
ffead-cpp-postgresql16,325
2.4%
FulC++ffeffeLinRea0
cppcms-postgres16,186
2.4%
PltC++NonNonLinRea0
cppcms15,479
2.3%
PltC++NonNonLinRea0
ffead-cpp-nginx-mong14,634
2.1%
FulC++ffengxLinRea277,520
ffead-cpp-apache-mon12,853
1.9%
FulC++ffeApcLinRea377,071
tornado-postgresql-r12,705
1.9%
PltPyNonTorLinRea0
ffead-cpp12,012
1.8%
FulC++ffeffeLinRea600,563
flask-pypy211,777
1.7%
McrPyNonTorLinRea0
cppcms-direct11,366
1.7%
PltC++NonNonLinRea0
ninja-standalone10,946
1.6%
FulJavJtyNonLinRea0
bottle-nginx-uwsgi4,517
0.7%
McrPyuWSngxLinRea0
flask-nginx-uwsgi4,346
0.6%
McrPyNonngxLinRea0
weppy-nginx-uwsgi4,340
0.6%
FulPyuWSngxLinRea0
klein822
0.1%
McrPyNonTwiLinRea0
webware755
0.1%
McrPyNonNonLinRea0
cherrypy-py3484
0.1%
McrPyNonNonLinRea0
cherrypy300
0.0%
McrPyNonNonLinRea201
ffead-cpp-apache-mys0
0.0%
FulC++ffeApcLinRea0
ffead-cpp-mysql0
0.0%
FulC++ffeffeLinRea0
ffead-cpp-nginx-mysq0
0.0%
FulC++ffengxLinRea0
lunaDid not completeFulC++NonNonLinRea
luna-epollDid not completeFulC++NonNonLinRea
luna-thread-per-connDid not completeFulC++NonNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
JSON responses per second, Dell servers at ServerCentral
Framework163264128256512BestClsLngPltFEAosIAErrors
aah27,10025,10426,11633,35736,15837,37737,377FulGoNonNonLinRea0
actframework119,650188,778241,106317,891365,406389,125389,125FulJavUtwNonLinRea0
activeweb32,55249,29054,22357,29953,23946,88457,299FulJavSvtNonLinRea0
activeweb-jackson101,244161,550134,448129,331130,644128,265161,550FulJavSvtNonLinRea0
aiohttp38,45871,735118,526154,784174,316182,406182,406McrPyasyGunLinRea0
akka-http25,80747,13290,43295,69699,798102,259102,259McrScaAkkNonLinRea0
apistar99,622185,460272,954395,164322,347575,507575,507McrPyNonMeiLinRea0
api_hour24,58945,12379,79093,12299,088100,836100,836McrPyasyGunLinRea0
blade108,066162,868213,200302,773327,933324,139327,933FulJavNonNonLinRea0
blaze131,135222,745301,708460,695586,170617,537617,537McrScablaNonLinRea0
bottle75,095137,640208,593253,584350,609364,428364,428McrPyMeiNonLinRea0
bottle-nginx-uwsgi4,2824,4104,5174,4474,2324,1674,517McrPyuWSngxLinRea0
bottle-py364,303124,305199,251252,690293,097298,316298,316McrPyMeiNonLinRea0
bottle-pypy220,84536,29940,53930,47043,33043,72643,726McrPyTorNonLinRea0
colossus124,403199,263310,918305,589313,324497,039497,039McrScaAkkNonLinRea0
cpoll_cppsp157,112264,756386,963241,145646,776608,473646,776PltC++NonNonLinRea0
curacao48,23695,375109,121107,083107,178109,172109,172McrJavSvtNonLinRea5
cutelyst16,17022,14836,96650,36870,35293,02593,025FulC++QtNonLinRea0
cutelyst-nginx35,05567,55098,74086,14184,37483,52198,740FulC++QtngxLinRea0
cutelyst-pf-epoll22,32925,21615,67093,55590,25174,45093,555FulC++QtNonLinRea0
cutelyst-thread104,103188,941281,529405,116313,558611,406611,406FulC++QtNonLinRea0
cutelyst-thread-epol109,589202,258313,176220,329559,997220,257559,997FulC++QtNonLinRea0
cutelyst-thread-epol108,972186,039283,308333,249461,957220,518461,957FulC++QtNonLinRea0
django22,71841,95970,54277,27874,97371,12977,278FulPyNonMeiLinRea0
django-py336,80268,452115,326137,821146,354145,614146,354FulPyNonMeiLinRea0
dropwizard34,45040,01542,82643,50543,91144,60744,607FulJavJAXJtyLinRea0
falcon79,436155,253249,605224,089344,728477,082477,082McrPyNonMeiLinRea0
falcon-py380,510149,309235,279311,946353,057424,771424,771McrPyNonMeiLinRea0
falcon-pypy241,93170,95397,814131,950179,668199,972199,972McrPyNonTorLinRea0
falcore89,877138,785155,612213,491224,361229,102229,102McrGoNonNonLinRea0
ffead-cpp12,0120000012,012FulC++ffeffeLinRea600,563
ffead-cpp-apache-mon12,8530000012,853FulC++ffeApcLinRea377,071
ffead-cpp-apache-mys0000000FulC++ffeApcLinRea0
ffead-cpp-apache-pos16,81517,05316,90716,60216,25916,05117,053FulC++ffeApcLinRea0
ffead-cpp-mysql0000000FulC++ffeffeLinRea0
ffead-cpp-nginx-mong14,6340000014,634FulC++ffengxLinRea277,520
ffead-cpp-nginx-mysq0000000FulC++ffengxLinRea0
ffead-cpp-nginx-post16,58216,24516,02116,30316,19615,96116,582FulC++ffengxLinRea0
ffead-cpp-postgresql16,3250000016,325FulC++ffeffeLinRea0
finagle96,193169,883254,018311,887370,170420,556420,556McrScaNtyNonLinRea0
finatra100,474185,679301,036388,764462,553523,712523,712McrScaNtyNonLinRea0
finch93,950161,438262,923332,795375,221396,796396,796McrScaNtyNonLinRea0
fintrospect90,821168,566278,878372,977433,910504,706504,706McrScaNtyNonLinRea0
flask22,51344,51671,81980,33283,23080,73183,230McrPyMeiNonLinRea0
flask-nginx-uwsgi4,3464,1924,2364,2094,2664,2374,346McrPyNonngxLinRea0
flask-py323,23243,21079,35093,16393,91892,05793,918McrPyMeiNonLinRea0
flask-pypy26,23910,82611,77711,2707,8297,19411,777McrPyNonTorLinRea0
gemini139,945220,579279,808213,162498,666360,890498,666FulJavSvtResLinRea0
gin58,84695,670148,032213,352237,096282,655282,655McrGoNonNonLinRea0
goji53,45682,01448,57824,19223,65623,85182,014McrGoNonNonLinRea0
grizzly126,373222,082337,596273,515587,363641,988641,988McrJavSvtNonLinRea0
grizzly-jersey67,143119,970155,625206,787232,048286,502286,502McrJavJAXNonLinRea0
http4s47,82582,05982,65280,88680,42581,80882,652McrScaNIOblaLinRea0
japronto114,871207,914274,505223,871374,369629,566629,566McrPyNonNonLinRea0
jawn53,48052,414125,468138,805159,717162,837162,837FulJavSvtUtwLinRea0
undertow-jersey-c3p050,85053,966116,212140,923144,038145,981145,981PltJavJAXNonLinRea0
undertow-jersey-hika48,84851,366103,878142,545146,497145,140146,497PltJavJAXNonLinRea0
jooby74,00186,02093,22196,390101,33099,181101,330FulJavNtyNonLinRea0
kami60,72099,972170,014251,574268,516309,291309,291McrGoNonNonLinRea0
klein822816820819807796822McrPyNonTwiLinRea0
lunaDid not completeFulC++NonNonLinRea
luna-epollDid not completeFulC++NonNonLinRea
luna-thread-per-connDid not completeFulC++NonNonLinRea
minijax46,02152,21753,31653,20053,90752,84153,907FulJavJAXJtyLinRea0
morepath48,18194,207162,028196,807219,172218,625219,172McrPyMeiGunLinRea0
netty155,942259,002357,255219,667590,560589,815590,560PltJavNtyNonLinRea0
ninja-standalone9,1458,9438,9779,21310,83710,94610,946FulJavJtyNonLinRea0
bayou127,662139,946124,698122,035132,323152,958152,958PltJavBayNonLinRea0
beego55,45792,391133,949201,432226,290271,747271,747FulGoNonNonLinRea0
cherrypy291290288290300299300McrPyNonNonLinRea201
cherrypy-py3484481469467468469484McrPyNonNonLinRea0
comsat-servlet32,49142,25936,63533,42555,58336,28255,583PltJavSvtNonLinRea0
comsat-servlet-under30,63623,92117,43118,57316,12015,66030,636PltJavSvtNonLinRea0
comsat-webactors47,24274,24653,81234,88472,32948,79474,246PltJavComNonLinRea0
comsat-webactors-und54,15583,40959,22897,80156,72976,23697,801PltJavComNonLinRea0
cppcms15,47915,14215,16115,21315,20915,20915,479PltC++NonNonLinRea0
cppcms-direct11,36611,09211,22611,26211,27211,27111,366PltC++NonNonLinRea0
cppcms-postgres16,18615,94515,96315,60315,75415,91416,186PltC++NonNonLinRea0
echo58,78585,735125,455138,014138,023139,626139,626McrGoNonNonLinRea0
fasthttp129,812148,221174,704258,719310,145357,675357,675PltGoNonNonLinRea0
go61,80898,620153,752222,346248,717296,356296,356PltGoNonNonLinRea0
go-prefork109,268188,295284,100243,081436,384603,592603,592PltGoNonNonLinRea0
jetty61,20099,346163,217177,118166,748161,382177,118PltJavJtyNonLinRea0
jetty-servlet57,84895,857167,820182,147179,944177,650182,147PltJavJtyNonLinRea0
jlhttp151,273246,542360,176323,820315,819311,241360,176PltJavJLHNonLinRea0
light-4j137,299232,593352,417276,551634,889634,167634,889PltJavLigNonLinRea0
revel49,39684,421112,802109,83399,115102,555112,802FulGoNonNonLinRea0
servlet163,221250,186308,237500,631354,718657,791657,791PltJavSvtResLinRea0
servlet-afterburner164,924260,427354,688217,628434,911662,134662,134PltJavSvtResLinRea0
servlet-cjs164,149245,857267,987350,563626,197663,623663,623PltJavSvtResLinRea0
tornado15,71929,78549,95559,29359,22459,08859,293PltPyNonTorLinRea0
tornado-postgresql-r12,57012,59412,70512,59812,51612,46512,705PltPyNonTorLinRea0
tornado-py313,64726,00743,15353,53752,97752,95153,537PltPyNonTorLinRea0
tornado-pypy253,02793,975144,063193,115196,384195,033196,384PltPyNonTorLinRea0
undertow145,048226,868271,403491,805354,520595,431595,431PltJavUtwNonLinRea0
vertx137,383229,785331,541445,892513,059550,381550,381PltJavVerNonLinRea0
wildfly-ee732,85830,26053,69665,50862,58870,10370,103FulJavSvtWilLinRea0
play122,37331,95744,11755,26652,67951,83655,266FulJavNtyNonLinRea0
play236,16438,86150,77057,18458,04356,84958,043FulJavAkkNonLinRea0
play2-java-netty57,51895,693129,018171,522176,509174,205176,509FulJavNtyNonLinRea0
play2-scala53,95780,54293,00291,75190,16788,81093,002FulScaAkkNonLinRea0
play2-scala-scala-ne68,860119,708191,512208,400185,559108,423208,400FulScaNtyNonLinRea0
proteus130,635218,428311,447343,015592,389658,410658,410McrJavUtwNonLinRea0
pyramid36,85372,815117,230146,094157,668157,687157,687FulPyNonMeiLinRea0
pyramid-py241,47779,526135,451161,736174,835175,201175,201FulPyNonMeiLinRea0
rapidoid154,597257,741303,439435,543229,629662,269662,269PltJavRapNonLinRea0
rapidoid-http-fast158,925186,982334,638225,098633,509642,181642,181PltJavRapNonLinRea0
restexpress32,10853,759115,937115,157115,250114,522115,937McrJavNtyNonLinRea0
revenj-jvm166,623247,393380,956239,216569,099669,526669,526FulJavSvtResLinRea0
s-server148,541230,120206,888360,200633,811561,419633,811PltScas-sNonLinRea0
sanic43,20897,073154,921205,390204,900219,254219,254McrPyNonNonLinRea0
scruffy23,01931,89643,84754,18648,09246,54254,186McrScaUtwNonLinRea0
silicon124,680156,779133,132134,027134,691133,617156,779McrC++NonmicLinRea0
spark75,127132,965180,744225,691203,579200,415225,691McrJavSvtResLinRea0
spray34,929107,382108,327131,403139,686148,368148,368McrScaAkkNonLinRea0
spring26,11425,52524,41923,17623,36523,50226,114FulJavUtwNonLinRea0
tapestry67,441119,881146,931135,610133,409127,889146,931FulJavSvtResLinRea0
treefrog79,063148,879238,267304,265219,226221,419304,265FulC++NonNonLinRea0
treefrog-hybrid21,22932,19143,40356,44565,80769,68169,681FulC++NonNonLinRea18
treefrog-mongodb78,362149,258251,523306,788233,016243,818306,788FulC++NonNonLinRea0
treefrog-postgres78,448147,887246,267293,190256,231276,258293,190FulC++NonNonLinRea0
turbogears24,31840,33979,67286,53579,57477,01486,535McrPyNonMeiLinRea0
ulib-json175,758286,111363,723284,583618,440682,383682,383PltC++NonULiLinRea0
ulib-json_fit182,215291,335251,618557,083265,877664,458664,458PltC++NonULiLinRea0
uvicorn80,604153,377242,204329,965313,032475,636475,636PltPyNonNonLinRea0
vertx-web126,694221,640331,125289,053548,259570,243570,243McrJavvtxNonLinRea0
web2py7,35613,97620,83420,84819,80519,83620,848FulPyMeiNonLinRea0
web2py-optimized16,19231,34247,34650,93347,02446,21750,933FulPyMeiNonLinRea0
webgo53,91796,254131,116139,313136,670137,543139,313McrGoNonNonLinRea0
webware755743737734740734755McrPyNonNonLinRea0
weppy52,438106,094176,706210,161249,253211,690249,253FulPyMeiNonLinRea0
weppy-nginx-uwsgi4,1414,2044,3404,3264,1644,1014,340FulPyuWSngxLinRea0
weppy-py352,46296,732172,127215,281158,143218,228218,228FulPyMeiNonLinRea0
weppy-pypy215,40535,91554,64050,12458,87862,79062,790FulPyTorNonLinRea0
wheezyweb79,557145,573235,502301,839380,972387,714387,714McrPyNonMeiLinRea0
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
wicket104,203168,138228,476265,298283,711268,136283,711FulJavSvtResLinRea0
wt13,41913,50738,63859,38558,06657,71759,385FulC++NonNonLinRea0
Latency of JSON responses, Dell servers at ServerCentral
FrameworkAverage latency (lower is better)σ (SD)MaxErrors
cutelyst-thread0.8 ms
0.1%
0.6 ms28.7 ms0
ulib-json_fit0.9 ms
0.1%
0.9 ms25.1 ms0
cpoll_cppsp0.9 ms
0.1%
0.8 ms17.8 ms0
ulib-json0.9 ms
0.1%
1.3 ms29.6 ms0
s-server1.1 ms
0.1%
1.7 ms44.9 ms0
light-4j1.2 ms
0.1%
2.7 ms87.9 ms0
proteus1.2 ms
0.1%
4.3 ms206.3 ms0
colossus1.2 ms
0.1%
1.7 ms44.1 ms0
rapidoid-http-fast1.4 ms
0.1%
7.2 ms203.5 ms0
vertx1.4 ms
0.1%
2.4 ms48.4 ms0
undertow1.4 ms
0.1%
5.8 ms279.6 ms0
revenj-jvm1.4 ms
0.1%
8.2 ms212.1 ms0
servlet-afterburner1.4 ms
0.1%
6.4 ms204.5 ms0
netty1.5 ms
0.1%
6.5 ms209.0 ms0
grizzly1.5 ms
0.1%
8.7 ms232.0 ms0
japronto1.5 ms
0.1%
10.4 ms410.1 ms0
servlet-cjs1.5 ms
0.1%
8.2 ms212.4 ms0
servlet1.5 ms
0.1%
7.8 ms210.7 ms0
go-prefork1.6 ms
0.1%
9.2 ms212.0 ms0
rapidoid1.6 ms
0.1%
9.4 ms212.9 ms0
apistar1.6 ms
0.1%
9.9 ms211.5 ms0
fasthttp1.7 ms
0.2%
7.0 ms227.3 ms0
uvicorn1.7 ms
0.2%
8.9 ms230.1 ms0
falcon1.8 ms
0.2%
9.1 ms217.3 ms0
kami1.9 ms
0.2%
2.1 ms55.3 ms0
falcon-py32.1 ms
0.2%
9.9 ms212.3 ms0
go2.2 ms
0.2%
2.6 ms45.3 ms0
cutelyst-thread-epol2.3 ms
0.2%
0.9 ms10.7 ms0
cutelyst-thread-epol2.3 ms
0.2%
1.1 ms25.2 ms0
beego2.5 ms
0.2%
3.0 ms54.8 ms0
gin2.6 ms
0.2%
7.1 ms212.9 ms0
vertx-web2.8 ms
0.3%
12.6 ms259.7 ms0
blade3.0 ms
0.3%
4.2 ms65.2 ms0
weppy-py33.1 ms
0.3%
8.5 ms222.1 ms0
jawn3.4 ms
0.3%
3.6 ms205.8 ms0
wheezyweb3.4 ms
0.3%
11.6 ms376.9 ms0
aiohttp3.5 ms
0.3%
10.5 ms221.3 ms0
sanic3.6 ms
0.3%
13.7 ms423.2 ms0
echo3.7 ms
0.3%
1.6 ms40.4 ms0
weppy3.8 ms
0.3%
10.8 ms323.2 ms0
actframework3.9 ms
0.3%
8.4 ms209.7 ms0
bottle4.2 ms
0.4%
12.1 ms430.2 ms0
bottle-py34.4 ms
0.4%
11.2 ms359.8 ms0
falcon-pypy24.4 ms
0.4%
14.5 ms397.8 ms0
tornado-pypy24.5 ms
0.4%
14.5 ms447.0 ms0
undertow-jersey-c3p04.5 ms
0.4%
9.0 ms245.8 ms0
treefrog-postgres4.6 ms
0.4%
31.2 ms837.9 ms0
falcore4.8 ms
0.4%
8.8 ms246.8 ms0
spray5.1 ms
0.5%
17.1 ms426.7 ms0
treefrog5.1 ms
0.5%
27.4 ms814.9 ms0
undertow-jersey-hika5.2 ms
0.5%
12.5 ms305.1 ms0
jlhttp5.3 ms
0.5%
11.9 ms409.3 ms0
cutelyst5.6 ms
0.5%
2.4 ms26.6 ms0
morepath5.8 ms
0.5%
13.5 ms477.3 ms0
api_hour6.0 ms
0.5%
12.5 ms414.3 ms0
finch6.1 ms
0.5%
25.6 ms828.6 ms0
jooby6.1 ms
0.5%
7.6 ms119.7 ms0
webgo6.4 ms
0.6%
8.4 ms233.0 ms0
blaze6.4 ms
0.6%
56.0 ms1600.0 ms0
spark7.3 ms
0.7%
12.0 ms223.4 ms0
pyramid7.5 ms
0.7%
16.5 ms435.2 ms0
revel7.7 ms
0.7%
10.5 ms262.5 ms0
cutelyst-nginx7.8 ms
0.7%
8.2 ms121.1 ms0
activeweb-jackson7.8 ms
0.7%
11.8 ms222.9 ms0
fintrospect8.1 ms
0.7%
66.0 ms1640.0 ms0
finatra8.3 ms
0.7%
39.9 ms862.3 ms0
curacao8.3 ms
0.7%
18.8 ms520.8 ms5
bayou8.5 ms
0.8%
51.8 ms1610.0 ms0
cutelyst-pf-epoll8.9 ms
0.8%
10.1 ms152.2 ms0
pyramid-py29.3 ms
0.8%
20.6 ms533.3 ms0
tornado9.3 ms
0.8%
12.0 ms379.2 ms0
treefrog-mongodb9.7 ms
0.9%
71.2 ms1620.0 ms0
tornado-py39.9 ms
0.9%
10.3 ms410.5 ms0
flask-py39.9 ms
0.9%
19.7 ms638.9 ms0
django-py310.3 ms
0.9%
23.9 ms717.9 ms0
finagle10.4 ms
0.9%
66.5 ms1690.0 ms0
play2-java-netty10.7 ms
1.0%
31.3 ms512.2 ms0
restexpress10.9 ms
1.0%
44.3 ms1040.0 ms0
grizzly-jersey11.5 ms
1.0%
40.1 ms476.6 ms0
dropwizard11.5 ms
1.0%
3.7 ms81.2 ms0
jetty-servlet11.5 ms
1.0%
76.6 ms1640.0 ms0
treefrog-hybrid11.8 ms
1.1%
55.6 ms1640.0 ms18
gemini11.9 ms
1.1%
75.1 ms910.4 ms0
wicket12.0 ms
1.1%
24.3 ms343.1 ms0
flask13.3 ms
1.2%
23.7 ms478.0 ms0
silicon14.0 ms
1.2%
96.1 ms1670.0 ms0
turbogears14.0 ms
1.3%
26.3 ms689.0 ms0
jetty14.3 ms
1.3%
69.0 ms1710.0 ms0
wt14.3 ms
1.3%
48.7 ms933.5 ms0
django15.0 ms
1.3%
26.5 ms616.2 ms0
comsat-webactors-und16.5 ms
1.5%
68.9 ms635.8 ms0
minijax16.7 ms
1.5%
71.8 ms1650.0 ms0
wildfly-ee717.3 ms
1.5%
28.8 ms415.3 ms0
play117.6 ms
1.6%
81.5 ms1610.0 ms0
http4s18.1 ms
1.6%
102.2 ms1600.0 ms0
web2py-optimized18.1 ms
1.6%
26.2 ms428.2 ms0
cherrypy-py321.3 ms
1.9%
5.1 ms233.8 ms0
goji22.5 ms
2.0%
18.0 ms129.7 ms0
play2-scala22.8 ms
2.0%
89.3 ms1470.0 ms0
spring24.4 ms
2.2%
19.3 ms230.7 ms0
activeweb24.6 ms
2.2%
44.0 ms568.0 ms0
tapestry24.9 ms
2.2%
40.3 ms428.7 ms0
ffead-cpp-nginx-post31.5 ms
2.8%
2.8 ms57.5 ms0
ffead-cpp-apache-pos31.6 ms
2.8%
2.6 ms60.2 ms0
cppcms-postgres32.2 ms
2.9%
3.3 ms41.2 ms0
aah33.1 ms
3.0%
73.7 ms1080.0 ms0
cherrypy33.3 ms
3.0%
7.1 ms253.0 ms201
weppy-pypy233.3 ms
3.0%
76.3 ms1610.0 ms0
cppcms33.6 ms
3.0%
2.5 ms40.5 ms0
play2-scala-scala-ne34.0 ms
3.0%
56.4 ms595.4 ms0
bottle-pypy234.8 ms
3.1%
74.8 ms1360.0 ms0
akka-http36.3 ms
3.2%
118.0 ms1300.0 ms0
web2py38.9 ms
3.5%
53.4 ms798.7 ms0
tornado-postgresql-r39.5 ms
3.5%
9.0 ms235.4 ms0
cppcms-direct44.2 ms
3.9%
4.6 ms247.0 ms0
play252.2 ms
4.7%
248.6 ms3300.0 ms0
ninja-standalone54.6 ms
4.9%
97.3 ms1700.0 ms0
flask-pypy295.9 ms
8.6%
119.4 ms1390.0 ms0
scruffy108.6 ms
9.7%
452.1 ms4890.0 ms0
flask-nginx-uwsgi120.3 ms
10.7%
14.3 ms361.3 ms0
bottle-nginx-uwsgi123.0 ms
11.0%
14.3 ms361.9 ms0
comsat-servlet-under124.9 ms
11.1%
203.7 ms1170.0 ms0
weppy-nginx-uwsgi125.1 ms
11.2%
15.0 ms387.7 ms0
comsat-webactors545.9 ms
48.7%
1030.0 ms4430.0 ms0
klein599.2 ms
53.5%
256.0 ms2890.0 ms0
webware681.4 ms
60.8%
82.1 ms734.5 ms0
comsat-servlet1120.0 ms
100.0%
1960.0 ms7580.0 ms0
Framework overhead of JSON responses, Dell servers at ServerCentral
Framework bestvsPlatform bestRatio (higher is better)
tornado-pypy2196,384vstornado59,293
331.2%
go-prefork603,592vsgo296,356
203.7%
finatra523,712vsfinagle420,556
124.5%
fasthttp357,675vsgo296,356
120.7%
fintrospect504,706vsfinagle420,556
120.0%
kami309,291vsgo296,356
104.4%
vertx-web570,243vsvertx550,381
103.6%
revenj-jvm669,526vsservlet657,791
101.8%
servlet-cjs663,623vsservlet657,791
100.9%
treefrog-mongodb306,788vstreefrog304,265
100.8%
servlet-afterburner662,134vsservlet657,791
100.7%
undertow-jersey-hika146,497vsundertow-jersey-c3p0145,981
100.4%
undertow-jersey-c3p0145,981vsundertow-jersey-hika146,497
99.6%
gin282,655vsgo296,356
95.4%
finch396,796vsfinagle420,556
94.4%
beego271,747vsgo296,356
91.7%
play2-scala93,002vsakka-http102,259
90.9%
tornado-py353,537vstornado59,293
90.3%
falcon477,082vswsgi604,690
78.9%
falcore229,102vsgo296,356
77.3%
gemini498,666vsservlet657,791
75.8%
finagle420,556vsnetty590,560
71.2%
falcon-py3424,771vswsgi604,690
70.2%
actframework389,125vsundertow595,431
65.4%
bottle364,428vswsgi604,690
60.3%
play258,043vsakka-http102,259
56.8%
bottle-py3298,316vswsgi604,690
49.3%
echo139,626vsgo296,356
47.1%
webgo139,313vsgo296,356
47.0%
wicket283,711vsservlet657,791
43.1%
weppy249,253vswsgi604,690
41.2%
revel112,802vsgo296,356
38.1%
weppy-py3218,228vswsgi604,690
36.1%
play2-scala-scala-ne208,400vsnetty590,560
35.3%
spark225,691vsservlet657,791
34.3%
falcon-pypy2199,972vswsgi604,690
33.1%
play2-java-netty176,509vsnetty590,560
29.9%
pyramid-py2175,201vswsgi604,690
29.0%
jetty-servlet182,147vsservlet657,791
27.7%
goji82,014vsgo296,356
27.7%
jetty177,118vsservlet657,791
26.9%
pyramid157,687vswsgi604,690
26.1%
jawn162,837vsservlet657,791
24.8%
activeweb-jackson161,550vsservlet657,791
24.6%
django-py3146,354vswsgi604,690
24.2%
treefrog-hybrid69,681vstreefrog304,265
22.9%
tapestry146,931vsservlet657,791
22.3%
tornado-postgresql-r12,705vstornado59,293
21.4%
restexpress115,937vsnetty590,560
19.6%
jooby101,330vsnetty590,560
17.2%
curacao109,172vsservlet657,791
16.6%
flask-py393,918vswsgi604,690
15.5%
flask83,230vswsgi604,690
13.8%
django77,278vswsgi604,690
12.8%
aah37,377vsgo296,356
12.6%
weppy-pypy262,790vswsgi604,690
10.4%
play155,266vsnetty590,560
9.4%
scruffy54,186vsundertow595,431
9.1%
activeweb57,299vsservlet657,791
8.7%
web2py-optimized50,933vswsgi604,690
8.4%
bottle-pypy243,726vswsgi604,690
7.2%
dropwizard44,607vsservlet657,791
6.8%
ninja-standalone10,946vsjetty177,118
6.2%
spring26,114vsservlet657,791
4.0%
web2py20,848vswsgi604,690
3.4%
flask-pypy211,777vswsgi604,690
1.9%
bottle-nginx-uwsgi4,517vswsgi604,690
0.7%
flask-nginx-uwsgi4,346vswsgi604,690
0.7%
weppy-nginx-uwsgi4,340vswsgi604,690
0.7%
Best (bar chart)
Data table
Latency
Framework overhead
Best (bar chart)
Data table
Latency
Framework overhead

Requirements summary

In this test, each response is a JSON serialization of a freshly-instantiated object that maps the key message to the value Hello, World!

Example response:

HTTP/1.1 200 OK Content-Type: application/json Content-Length: 28 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"message":"Hello, World!"}

For a more detailed description of the requirements, see the Source Code and Requirements section.

Best plaintext responses per second, Dell servers at ServerCentral(134 tests)
FrameworkBest performance (higher is better)ClsLngPltFEAosIAErrors
ulib-plaintext_fit2,684,333
100.0%(98.0%)
PltC++NonULiLinRea2
ulib2,679,274
99.8%(97.8%)
PltC++NonULiLinRea2
fasthttp2,665,278
99.3%(97.3%)
PltGoNonNonLinRea0
rapidoid-http-fast2,654,663
98.9%(96.9%)
PltJavRapNonLinRea0
rapidoid2,608,986
97.2%(95.3%)
PltJavRapNonLinRea0
undertow2,544,955
94.8%(92.9%)
PltJavUtwNonLinRea0
vertx2,503,741
93.3%(91.4%)
PltJavVerNonLinRea0
netty2,447,315
91.2%(89.4%)
PltJavNtyNonLinRea1
proteus2,441,688
91.0%(89.2%)
McrJavUtwNonLinRea0
light-4j2,435,150
90.7%(88.9%)
PltJavLigNonLinRea0
actframework2,424,671
90.3%(88.5%)
FulJavUtwNonLinRea0
vertx-web2,411,118
89.8%(88.0%)
McrJavvtxNonLinRea0
japronto2,356,964
87.8%(86.1%)
McrPyNonNonLinRea178
cpoll_cppsp2,119,565
79.0%(77.4%)
PltC++NonNonLinRea0
bayou1,752,492
65.3%(64.0%)
PltJavBayNonLinRea4,674
cutelyst-thread-epol1,625,125
60.5%(59.3%)
FulC++QtNonLinRea0
s-server1,560,315
58.1%(57.0%)
PltScas-sNonLinRea0
colossus1,440,236
53.7%(52.6%)
McrScaAkkNonLinRea0
blaze1,362,644
50.8%(49.8%)
McrScablaNonLinRea2,638
cutelyst-thread1,032,448
38.5%(37.7%)
FulC++QtNonLinRea0
go-prefork1,011,589
37.7%(36.9%)
PltGoNonNonLinRea0
grizzly965,831
36.0%(35.3%)
McrJavSvtNonLinRea0
cutelyst-thread-epol938,400
35.0%(34.3%)
FulC++QtNonLinRea17
apistar931,909
34.7%(34.0%)
McrPyNonMeiLinRea0
revenj-jvm884,318
32.9%(32.3%)
FulJavSvtResLinRea0
gemini824,347
30.7%(30.1%)
FulJavSvtResLinRea0
wheezyweb824,132
30.7%(30.1%)
McrPyNonMeiLinRea0
go752,853
28.0%(27.5%)
PltGoNonNonLinRea0
gin750,780
28.0%(27.4%)
McrGoNonNonLinRea0
servlet730,095
27.2%(26.7%)
PltJavSvtResLinRea0
finatra710,811
26.5%(26.0%)
McrScaNtyNonLinRea3,358
falcon695,897
25.9%(25.4%)
McrPyNonMeiLinRea0
kami686,682
25.6%(25.1%)
McrGoNonNonLinRea0
uvicorn624,188
23.3%(22.8%)
PltPyNonNonLinRea0
falcon-py3610,525
22.7%(22.3%)
McrPyNonMeiLinRea0
beego604,659
22.5%(22.1%)
FulGoNonNonLinRea0
finagle556,888
20.7%(20.3%)
McrScaNtyNonLinRea754
fintrospect548,336
20.4%(20.0%)
McrScaNtyNonLinRea782
restexpress497,486
18.5%(18.2%)
McrJavNtyNonLinRea7,071
blade490,441
18.3%(17.9%)
FulJavNonNonLinRea0
akka-http490,038
18.3%(17.9%)
McrScaAkkNonLinRea2,254
finch477,543
17.8%(17.4%)
McrScaNtyNonLinRea3,742
bottle462,141
17.2%(16.9%)
McrPyMeiNonLinRea0
bottle-py3406,822
15.2%(14.9%)
McrPyMeiNonLinRea0
jlhttp386,636
14.4%(14.1%)
PltJavJLHNonLinRea646
treefrog-hybrid382,951
14.3%(14.0%)
FulC++NonNonLinRea539
weppy365,421
13.6%(13.3%)
FulPyMeiNonLinRea0
weppy-py3363,672
13.5%(13.3%)
FulPyMeiNonLinRea0
pyramid-py2331,732
12.4%(12.1%)
FulPyNonMeiLinRea0
falcore331,164
12.3%(12.1%)
McrGoNonNonLinRea0
pyramid310,715
11.6%(11.3%)
FulPyNonMeiLinRea0
jetty305,109
11.4%(11.1%)
PltJavJtyNonLinRea584
jooby294,958
11.0%(10.8%)
FulJavNtyNonLinRea0
spray294,030
11.0%(10.7%)
McrScaAkkNonLinRea0
wicket271,565
10.1%(9.9%)
FulJavSvtResLinRea0
jetty-servlet262,519
9.8%(9.6%)
PltJavJtyNonLinRea4,788
morepath256,610
9.6%(9.4%)
McrPyMeiGunLinRea0
minijax226,826
8.4%(8.3%)
FulJavJAXJtyLinRea4,270
play2-java-netty223,814
8.3%(8.2%)
FulJavNtyNonLinRea0
falcon-pypy2221,939
8.3%(8.1%)
McrPyNonTorLinRea0
play2-scala221,514
8.3%(8.1%)
FulScaAkkNonLinRea0
treefrog-postgres219,856
8.2%(8.0%)
FulC++NonNonLinRea2,519
jawn219,577
8.2%(8.0%)
FulJavSvtUtwLinRea0
treefrog-mongodb217,082
8.1%(7.9%)
FulC++NonNonLinRea40
treefrog213,423
8.0%(7.8%)
FulC++NonNonLinRea43
tornado-pypy2201,831
7.5%(7.4%)
PltPyNonTorLinRea0
aiohttp191,436
7.1%(7.0%)
McrPyasyGunLinRea0
grizzly-jersey190,532
7.1%(7.0%)
McrJavJAXNonLinRea0
silicon172,714
6.4%(6.3%)
McrC++NonmicLinRea53,781
undertow-jersey-hika170,106
6.3%(6.2%)
PltJavJAXNonLinRea863
django-py3159,900
6.0%(5.8%)
FulPyNonMeiLinRea0
undertow-jersey-c3p0154,758
5.8%(5.7%)
PltJavJAXNonLinRea0
play2-scala-scala-ne154,544
5.8%(5.6%)
FulScaNtyNonLinRea0
echo150,523
5.6%(5.5%)
McrGoNonNonLinRea0
tapestry136,018
5.1%(5.0%)
FulJavSvtResLinRea0
spring131,249
4.9%(4.8%)
FulJavUtwNonLinRea0
activeweb129,324
4.8%(4.7%)
FulJavSvtNonLinRea0
api_hour123,385
4.6%(4.5%)
McrPyasyGunLinRea0
flask-py3121,901
4.5%(4.5%)
McrPyMeiNonLinRea0
comsat-servlet-under121,743
4.5%(4.4%)
PltJavSvtNonLinRea0
curacao119,759
4.5%(4.4%)
McrJavSvtNonLinRea3
revel117,584
4.4%(4.3%)
FulGoNonNonLinRea0
flask113,578
4.2%(4.1%)
McrPyMeiNonLinRea0
wt111,114
4.1%(4.1%)
FulC++NonNonLinRea0
cutelyst106,310
4.0%(3.9%)
FulC++QtNonLinRea0
poco102,247
3.8%(3.7%)
PltC++NonNonLinRea233,647
comsat-servlet99,576
3.7%(3.6%)
PltJavSvtNonLinRea0
bottle-pypy295,722
3.6%(3.5%)
McrPyTorNonLinRea0
dropwizard94,713
3.5%(3.5%)
FulJavJAXJtyLinRea0
turbogears88,620
3.3%(3.2%)
McrPyNonMeiLinRea0
http4s87,193
3.2%(3.2%)
McrScaNIOblaLinRea931
cutelyst-nginx85,448
3.2%(3.1%)
FulC++QtngxLinRea0
comsat-webactors-und81,287
3.0%(3.0%)
PltJavComNonLinRea0
django77,134
2.9%(2.8%)
FulPyNonMeiLinRea0
play270,167
2.6%(2.6%)
FulJavAkkNonLinRea1,911
wildfly-ee766,156
2.5%(2.4%)
FulJavSvtWilLinRea3
tornado61,738
2.3%(2.3%)
PltPyNonTorLinRea0
cutelyst-pf-epoll61,383
2.3%(2.2%)
FulC++QtNonLinRea0
tornado-py355,282
2.1%(2.0%)
PltPyNonTorLinRea0
comsat-webactors51,727
1.9%(1.9%)
PltJavComNonLinRea0
web2py-optimized50,373
1.9%(1.8%)
FulPyMeiNonLinRea0
aah44,783
1.7%(1.6%)
FulGoNonNonLinRea0
scruffy41,366
1.5%(1.5%)
McrScaUtwNonLinRea5,629
weppy-pypy222,764
0.8%(0.8%)
FulPyTorNonLinRea0
goji22,435
0.8%(0.8%)
McrGoNonNonLinRea0
web2py20,143
0.8%(0.7%)
FulPyMeiNonLinRea0
cppcms15,665
0.6%(0.6%)
PltC++NonNonLinRea0
cppcms-postgres15,597
0.6%(0.6%)
PltC++NonNonLinRea0
tornado-postgresql-r13,267
0.5%(0.5%)
PltPyNonTorLinRea0
ninja-standalone13,069
0.5%(0.5%)
FulJavJtyNonLinRea568
flask-pypy211,316
0.4%(0.4%)
McrPyNonTorLinRea0
cppcms-direct10,587
0.4%(0.4%)
PltC++NonNonLinRea3
flask-nginx-uwsgi4,868
0.2%(0.2%)
McrPyNonngxLinRea0
weppy-nginx-uwsgi4,752
0.2%(0.2%)
FulPyuWSngxLinRea0
bottle-nginx-uwsgi4,735
0.2%(0.2%)
McrPyuWSngxLinRea0
spark2,604
0.1%(0.1%)
McrJavSvtResLinRea0
klein781
0.0%(0.0%)
McrPyNonTwiLinRea26
webware730
0.0%(0.0%)
McrPyNonNonLinRea266
cherrypy-py3493
0.0%(0.0%)
McrPyNonNonLinRea699
cherrypy298
0.0%(0.0%)
McrPyNonNonLinRea598
sanic21
0.0%(0.0%)
McrPyNonNonLinRea656
ffead-cpp0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-apache-mon0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-apache-mys0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-apache-pos0
0.0%(0.0%)
FulC++ffeApcLinRea0
ffead-cpp-mysql0
0.0%(0.0%)
FulC++ffeffeLinRea0
ffead-cpp-nginx-mong0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-nginx-mysq0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-nginx-post0
0.0%(0.0%)
FulC++ffengxLinRea0
ffead-cpp-postgresql0
0.0%(0.0%)
FulC++ffeffeLinRea0
lunaDid not completeFulC++NonNonLinRea
luna-epollDid not completeFulC++NonNonLinRea
luna-thread-per-connDid not completeFulC++NonNonLinRea
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
Plaintext responses per second, Dell servers at ServerCentral
Framework2561,0244,09616,384BestClsLngPltFEAosIAErrors
aah44,78344,19644,49144,64744,783FulGoNonNonLinRea0
actframework1,883,7842,424,6712,172,0841,377,6262,424,671FulJavUtwNonLinRea0
activeweb126,963129,324110,178109,193129,324FulJavSvtNonLinRea0
aiohttp191,436183,896167,232160,864191,436McrPyasyGunLinRea0
akka-http381,176365,210483,034490,038490,038McrScaAkkNonLinRea2,254
apistar778,198931,909865,096856,448931,909McrPyNonMeiLinRea0
api_hour123,385119,163112,495112,229123,385McrPyasyGunLinRea0
blade471,308490,441485,280463,334490,441FulJavNonNonLinRea0
blaze993,687831,8251,324,3771,362,6441,362,644McrScablaNonLinRea2,638
bottle454,386462,141459,524452,469462,141McrPyMeiNonLinRea0
bottle-nginx-uwsgi4,7354,5874,1813,3124,735McrPyuWSngxLinRea0
bottle-py3395,648406,822390,775369,430406,822McrPyMeiNonLinRea0
bottle-pypy263,70181,25195,72294,06395,722McrPyTorNonLinRea0
colossus1,356,7451,440,2361,405,3691,319,7861,440,236McrScaAkkNonLinRea0
cpoll_cppsp1,694,3162,119,5652,112,0002,033,0282,119,565PltC++NonNonLinRea0
curacao108,118119,759107,77918,583119,759McrJavSvtNonLinRea3
cutelyst75,984105,287105,202106,310106,310FulC++QtNonLinRea0
cutelyst-nginx85,44882,47180,77679,98485,448FulC++QtngxLinRea0
cutelyst-pf-epoll56,61761,38355,80441,93761,383FulC++QtNonLinRea0
cutelyst-thread88,581353,6791,032,448567,7651,032,448FulC++QtNonLinRea0
cutelyst-thread-epol83,877351,9061,564,8831,625,1251,625,125FulC++QtNonLinRea0
cutelyst-thread-epol81,532347,672912,158938,400938,400FulC++QtNonLinRea17
django77,13473,99273,04771,96177,134FulPyNonMeiLinRea0
django-py3159,900159,665159,111155,396159,900FulPyNonMeiLinRea0
dropwizard75,71776,86888,26994,71394,713FulJavJAXJtyLinRea0
falcon638,629695,897690,542680,588695,897McrPyNonMeiLinRea0
falcon-py3601,449610,525609,501596,159610,525McrPyNonMeiLinRea0
falcon-pypy2210,755221,939218,011197,093221,939McrPyNonTorLinRea0
falcore280,107308,018331,164330,254331,164McrGoNonNonLinRea0
ffead-cpp00000FulC++ffeffeLinRea0
ffead-cpp-apache-mon00000FulC++ffeApcLinRea0
ffead-cpp-apache-mys00000FulC++ffeApcLinRea0
ffead-cpp-apache-pos00000FulC++ffeApcLinRea0
ffead-cpp-mysql00000FulC++ffeffeLinRea0
ffead-cpp-nginx-mong00000FulC++ffengxLinRea0
ffead-cpp-nginx-mysq00000FulC++ffengxLinRea0
ffead-cpp-nginx-post00000FulC++ffengxLinRea0
ffead-cpp-postgresql00000FulC++ffeffeLinRea0
finagle464,303556,888549,171527,786556,888McrScaNtyNonLinRea754
finatra654,644639,304710,811686,992710,811McrScaNtyNonLinRea3,358
finch426,744469,315477,543477,075477,543McrScaNtyNonLinRea3,742
fintrospect517,139548,336547,647538,744548,336McrScaNtyNonLinRea782
flask113,578112,443113,25239,929113,578McrPyMeiNonLinRea0
flask-nginx-uwsgi4,8684,8214,4263,5494,868McrPyNonngxLinRea0
flask-py3121,901120,890119,63147,396121,901McrPyMeiNonLinRea0
flask-pypy210,1009,97611,3167,26011,316McrPyNonTorLinRea0
gemini678,004824,347822,893808,414824,347FulJavSvtResLinRea0
gin605,826721,486750,780647,205750,780McrGoNonNonLinRea0
goji22,16321,06722,43521,57822,435McrGoNonNonLinRea0
grizzly840,803937,268965,831956,690965,831McrJavSvtNonLinRea0
grizzly-jersey163,269180,045190,532180,690190,532McrJavJAXNonLinRea0
http4s86,82987,19385,43085,36987,193McrScaNIOblaLinRea931
japronto2,305,0352,356,9642,280,1802,216,8092,356,964McrPyNonNonLinRea178
jawn219,577218,211197,206157,852219,577FulJavSvtUtwLinRea0
undertow-jersey-c3p0154,758149,289100,35860,094154,758PltJavJAXNonLinRea0
undertow-jersey-hika167,144155,908170,10685,950170,106PltJavJAXNonLinRea863
jooby294,958282,234274,329224,609294,958FulJavNtyNonLinRea0
kami643,562679,200686,682560,723686,682McrGoNonNonLinRea0
klein781715585682781McrPyNonTwiLinRea26
lunaDid not completeFulC++NonNonLinRea
luna-epollDid not completeFulC++NonNonLinRea
luna-thread-per-connDid not completeFulC++NonNonLinRea
minijax211,182215,942214,704226,826226,826FulJavJAXJtyLinRea4,270
morepath256,610253,937251,037120,685256,610McrPyMeiGunLinRea0
netty1,398,5372,426,2232,440,8692,447,3152,447,315PltJavNtyNonLinRea1
ninja-standalone13,03613,06912,67012,78613,069FulJavJtyNonLinRea568
bayou1,608,3781,732,0661,752,4921,670,0741,752,492PltJavBayNonLinRea4,674
beego470,819559,166604,659487,611604,659FulGoNonNonLinRea0
cherrypy298296271246298McrPyNonNonLinRea598
cherrypy-py3493481485444493McrPyNonNonLinRea699
comsat-servlet99,57640,06058,75893,78899,576PltJavSvtNonLinRea0
comsat-servlet-under107,193109,269121,74330,488121,743PltJavSvtNonLinRea0
comsat-webactors51,54531,67340,93751,72751,727PltJavComNonLinRea0
comsat-webactors-und81,28781,17614,75960,66581,287PltJavComNonLinRea0
cppcms15,66514,90715,41815,00515,665PltC++NonNonLinRea0
cppcms-direct10,57010,58710,2199,84210,587PltC++NonNonLinRea3
cppcms-postgres15,59714,59615,27914,74615,597PltC++NonNonLinRea0
echo148,912150,523147,825140,124150,523McrGoNonNonLinRea0
fasthttp2,665,2782,614,2652,582,6832,560,2062,665,278PltGoNonNonLinRea0
go527,944705,557752,853604,064752,853PltGoNonNonLinRea0
go-prefork698,192993,370999,1661,011,5891,011,589PltGoNonNonLinRea0
jetty280,633305,109300,796296,413305,109PltJavJtyNonLinRea584
jetty-servlet228,320234,322234,458262,519262,519PltJavJtyNonLinRea4,788
jlhttp364,312386,636373,849325,803386,636PltJavJLHNonLinRea646
light-4j1,930,1921,845,5832,418,0832,435,1502,435,150PltJavLigNonLinRea0
poco102,247101,307100,39898,599102,247PltC++NonNonLinRea233,647
revel96,623101,053105,350117,584117,584FulGoNonNonLinRea0
servlet569,319730,095719,057709,436730,095PltJavSvtResLinRea0
tornado61,54961,73857,77258,47461,738PltPyNonTorLinRea0
tornado-postgresql-r13,26713,08312,79411,50313,267PltPyNonTorLinRea0
tornado-py355,28254,79754,19652,19455,282PltPyNonTorLinRea0
tornado-pypy2201,831191,736180,891169,326201,831PltPyNonTorLinRea0
undertow1,953,5752,544,9552,525,0382,356,4752,544,955PltJavUtwNonLinRea0
vertx2,105,5622,381,3692,447,0222,503,7412,503,741PltJavVerNonLinRea0
wildfly-ee766,15657,86648,25731,24266,156FulJavSvtWilLinRea3
play267,77468,47470,16769,83970,167FulJavAkkNonLinRea1,911
play2-java-netty223,814137,476132,849131,022223,814FulJavNtyNonLinRea0
play2-scala221,514209,484211,909200,186221,514FulScaAkkNonLinRea0
play2-scala-scala-ne154,54419,14721,84521,036154,544FulScaNtyNonLinRea0
proteus1,733,1771,306,1662,376,0802,441,6882,441,688McrJavUtwNonLinRea0
pyramid310,715308,801305,968297,578310,715FulPyNonMeiLinRea0
pyramid-py2330,625331,732328,244320,306331,732FulPyNonMeiLinRea0
rapidoid2,567,8182,608,9862,557,0812,534,9792,608,986PltJavRapNonLinRea0
rapidoid-http-fast2,582,4582,654,6632,594,9302,571,4482,654,663PltJavRapNonLinRea0
restexpress412,690491,746497,486462,662497,486McrJavNtyNonLinRea7,071
revenj-jvm740,764884,318867,007814,061884,318FulJavSvtResLinRea0
s-server720,8751,504,8331,493,2331,560,3151,560,315PltScas-sNonLinRea0
sanic21143021McrPyNonNonLinRea656
scruffy3378,83726,17641,36641,366McrScaUtwNonLinRea5,629
silicon94,621163,551169,358172,714172,714McrC++NonmicLinRea53,781
spark2,4571,6261,6362,6042,604McrJavSvtResLinRea0
spray294,030277,515290,512283,708294,030McrScaAkkNonLinRea0
spring131,249130,717105,89449,498131,249FulJavUtwNonLinRea0
tapestry136,018129,375126,365131,974136,018FulJavSvtResLinRea0
treefrog213,423208,385210,597212,675213,423FulC++NonNonLinRea43
treefrog-hybrid346,224382,951370,906360,534382,951FulC++NonNonLinRea539
treefrog-mongodb217,082209,614209,445211,373217,082FulC++NonNonLinRea40
treefrog-postgres209,674208,639211,269219,856219,856FulC++NonNonLinRea2,519
turbogears88,18588,38888,62043,72388,620McrPyNonMeiLinRea0
ulib2,679,2742,666,0762,594,4032,571,7932,679,274PltC++NonULiLinRea2
ulib-plaintext_fit2,684,3332,643,6582,598,3132,573,6192,684,333PltC++NonULiLinRea2
uvicorn591,105624,188615,461580,966624,188PltPyNonNonLinRea0
vertx-web1,858,5482,411,1182,315,7992,221,3332,411,118McrJavvtxNonLinRea0
web2py20,14319,70717,4816,08520,143FulPyMeiNonLinRea0
web2py-optimized50,37349,69443,03810,95350,373FulPyMeiNonLinRea0
webware274728730164730McrPyNonNonLinRea266
weppy365,421328,929330,448319,389365,421FulPyMeiNonLinRea0
weppy-nginx-uwsgi4,7524,5714,1833,3924,752FulPyuWSngxLinRea0
weppy-py3322,562327,271363,672319,334363,672FulPyMeiNonLinRea0
weppy-pypy214,16120,06422,76419,60822,764FulPyTorNonLinRea0
wheezyweb652,390814,985824,132806,922824,132McrPyNonMeiLinRea0
wheezyweb-py3Did not completeMcrPyNonMeiLinRea
wicket185,102271,565264,242260,798271,565FulJavSvtResLinRea0
wt111,114109,294108,935108,385111,114FulC++NonNonLinRea0
Latency of plaintext responses, Dell servers at ServerCentral
FrameworkAverage latency (lower is better)σ (SD)MaxErrors
poco12.7 ms
0.3%
6.4 ms236.1 ms233,647
bayou28.1 ms
0.7%
42.2 ms1460.0 ms4,674
silicon46.6 ms
1.2%
14.8 ms641.5 ms53,781
japronto53.6 ms
1.4%
146.3 ms2300.0 ms178
vertx-web61.2 ms
1.5%
67.5 ms4090.0 ms0
finagle87.1 ms
2.2%
108.9 ms3570.0 ms754
colossus88.0 ms
2.2%
74.8 ms1980.0 ms0
uvicorn88.8 ms
2.2%
71.8 ms1550.0 ms0
treefrog-hybrid100.1 ms
2.5%
156.2 ms3530.0 ms539
cutelyst108.7 ms
2.8%
366.1 ms7250.0 ms0
go-prefork109.1 ms
2.8%
94.4 ms2720.0 ms0
fintrospect127.7 ms
3.2%
117.0 ms1670.0 ms782
finatra129.8 ms
3.3%
186.1 ms6490.0 ms3,358
finch145.9 ms
3.7%
179.1 ms3820.0 ms3,742
play2158.7 ms
4.0%
643.2 ms7880.0 ms1,911
ulib159.0 ms
4.0%
218.1 ms7710.0 ms2
cutelyst-thread167.6 ms
4.2%
571.3 ms7960.0 ms0
cutelyst-thread-epol173.1 ms
4.4%
593.9 ms7990.0 ms0
blaze175.0 ms
4.4%
409.8 ms7430.0 ms2,638
jetty-servlet186.4 ms
4.7%
114.5 ms2370.0 ms4,788
cutelyst-pf-epoll192.0 ms
4.9%
604.2 ms7120.0 ms0
apistar192.0 ms
4.9%
301.7 ms7020.0 ms0
gemini194.3 ms
4.9%
539.5 ms7780.0 ms0
spray196.1 ms
5.0%
496.3 ms7760.0 ms0
play2-scala196.7 ms
5.0%
768.2 ms8000.0 ms0
light-4j197.4 ms
5.0%
338.7 ms7890.0 ms0
cpoll_cppsp204.2 ms
5.2%
382.5 ms7980.0 ms0
s-server206.8 ms
5.2%
408.8 ms7850.0 ms0
vertx210.3 ms
5.3%
304.5 ms7760.0 ms0
ulib-plaintext_fit210.7 ms
5.3%
349.2 ms7990.0 ms2
treefrog-mongodb212.9 ms
5.4%
166.1 ms2340.0 ms40
proteus215.2 ms
5.4%
365.7 ms7900.0 ms0
rapidoid217.3 ms
5.5%
291.3 ms7380.0 ms0
wheezyweb218.1 ms
5.5%
229.6 ms4790.0 ms0
actframework236.3 ms
6.0%
278.3 ms5360.0 ms0
treefrog241.4 ms
6.1%
187.0 ms2590.0 ms43
grizzly244.0 ms
6.2%
914.6 ms7990.0 ms0
treefrog-postgres244.3 ms
6.2%
190.6 ms2890.0 ms2,519
rapidoid-http-fast244.5 ms
6.2%
339.9 ms7840.0 ms0
netty247.0 ms
6.3%
326.7 ms5240.0 ms1
revenj-jvm252.7 ms
6.4%
677.4 ms7780.0 ms0
jlhttp253.6 ms
6.4%
215.8 ms3580.0 ms646
akka-http256.1 ms
6.5%
794.9 ms7310.0 ms2,254
falcon257.2 ms
6.5%
394.9 ms7950.0 ms0
cutelyst-thread-epol261.5 ms
6.6%
804.9 ms7920.0 ms17
restexpress270.7 ms
6.9%
378.9 ms6670.0 ms7,071
falcon-py3272.6 ms
6.9%
339.4 ms7770.0 ms0
servlet284.1 ms
7.2%
840.5 ms7980.0 ms0
jetty290.8 ms
7.4%
306.0 ms3720.0 ms584
fasthttp295.0 ms
7.5%
454.4 ms7960.0 ms0
undertow322.8 ms
8.2%
477.9 ms7850.0 ms0
wt325.9 ms
8.2%
539.4 ms7980.0 ms0
wicket328.2 ms
8.3%
699.7 ms7920.0 ms0
bottle339.3 ms
8.6%
466.6 ms7880.0 ms0
cherrypy-py3360.6 ms
9.1%
154.9 ms1340.0 ms699
gin383.7 ms
9.7%
542.0 ms7550.0 ms0
tapestry388.5 ms
9.8%
803.7 ms7970.0 ms0
play2-java-netty396.0 ms
10.0%
872.0 ms7990.0 ms0
aiohttp397.3 ms
10.1%
319.1 ms2740.0 ms0
minijax425.9 ms
10.8%
449.5 ms7140.0 ms4,270
falcon-pypy2450.6 ms
11.4%
244.8 ms2110.0 ms0
tornado-pypy2459.5 ms
11.6%
240.5 ms2010.0 ms0
pyramid462.0 ms
11.7%
492.9 ms7640.0 ms0
blade478.5 ms
12.1%
699.8 ms7950.0 ms0
weppy480.6 ms
12.2%
420.8 ms7980.0 ms0
api_hour484.5 ms
12.3%
340.9 ms2780.0 ms0
bottle-py3487.8 ms
12.4%
571.3 ms7990.0 ms0
jooby488.0 ms
12.4%
288.9 ms2230.0 ms0
pyramid-py2489.6 ms
12.4%
531.2 ms7950.0 ms0
weppy-py3499.1 ms
12.6%
519.5 ms7560.0 ms0
http4s549.0 ms
13.9%
241.7 ms2480.0 ms931
kami557.4 ms
14.1%
876.8 ms7390.0 ms0
jawn623.5 ms
15.8%
417.7 ms7310.0 ms0
cherrypy649.4 ms
16.4%
462.9 ms2840.0 ms598
grizzly-jersey665.5 ms
16.8%
392.4 ms3170.0 ms0
play2-scala-scala-ne674.5 ms
17.1%
802.2 ms6880.0 ms0
spark674.5 ms
17.1%
1500.0 ms7730.0 ms0
activeweb745.7 ms
18.9%
1190.0 ms7990.0 ms0
falcore801.5 ms
20.3%
717.6 ms7940.0 ms0
django-py3857.7 ms
21.7%
720.2 ms7980.0 ms0
echo920.3 ms
23.3%
477.1 ms2520.0 ms0
go925.8 ms
23.4%
1340.0 ms8000.0 ms0
bottle-pypy2971.4 ms
24.6%
671.0 ms7460.0 ms0
beego1020.0 ms
25.8%
1450.0 ms8000.0 ms0
scruffy1030.0 ms
26.1%
1350.0 ms7260.0 ms5,629
morepath1060.0 ms
26.8%
1320.0 ms7970.0 ms0
comsat-servlet-under1240.0 ms
31.4%
409.1 ms3350.0 ms0
curacao1330.0 ms
33.7%
776.2 ms7220.0 ms3
revel1330.0 ms
33.7%
948.3 ms7780.0 ms0
tornado1370.0 ms
34.7%
664.0 ms4110.0 ms0
comsat-webactors-und1390.0 ms
35.2%
674.8 ms4210.0 ms0
django1590.0 ms
40.3%
1100.0 ms7800.0 ms0
comsat-webactors1620.0 ms
41.0%
2130.0 ms7940.0 ms0
turbogears1790.0 ms
45.3%
1460.0 ms7990.0 ms0
flask-py31810.0 ms
45.8%
1670.0 ms8000.0 ms0
cutelyst-nginx1840.0 ms
46.6%
1030.0 ms7100.0 ms0
undertow-jersey-hika1880.0 ms
47.6%
1550.0 ms7680.0 ms863
flask1930.0 ms
48.9%
1700.0 ms8000.0 ms0
web2py-optimized2010.0 ms
50.9%
1770.0 ms8000.0 ms0
comsat-servlet2030.0 ms
51.4%
1490.0 ms8000.0 ms0
aah2160.0 ms
54.7%
1610.0 ms8000.0 ms0
undertow-jersey-c3p02300.0 ms
58.2%
1610.0 ms6920.0 ms0
tornado-py32390.0 ms
60.5%
1300.0 ms8000.0 ms0
dropwizard2450.0 ms
62.0%
1750.0 ms8000.0 ms0
spring2690.0 ms
68.1%
1790.0 ms8000.0 ms0
ninja-standalone2890.0 ms
73.2%
1390.0 ms7810.0 ms568
cppcms-postgres3190.0 ms
80.8%
1960.0 ms8000.0 ms0
wildfly-ee73230.0 ms
81.8%
1780.0 ms8000.0 ms3
flask-pypy23330.0 ms
84.3%
2500.0 ms7980.0 ms0
weppy-pypy23470.0 ms
87.8%
1660.0 ms7980.0 ms0
cppcms3590.0 ms
90.9%
1730.0 ms7990.0 ms0
goji3650.0 ms
92.4%
1850.0 ms8000.0 ms0
klein3730.0 ms
94.4%
1600.0 ms7820.0 ms26
web2py3950.0 ms
100.0%
2020.0 ms8000.0 ms0
Framework overhead of plaintext responses, Dell servers at ServerCentral
Framework bestvsPlatform bestRatio (higher is better)
fasthttp2,665,278vsgo752,853
354.0%
tornado-pypy2201,831vstornado61,738
326.9%
treefrog-hybrid382,951vstreefrog213,423
179.4%
go-prefork1,011,589vsgo752,853
134.4%
finatra710,811vsfinagle556,888
127.6%
revenj-jvm884,318vsservlet730,095
121.1%
gemini824,347vsservlet730,095
112.9%
undertow-jersey-hika170,106vsundertow-jersey-c3p0154,758
109.9%
treefrog-mongodb217,082vstreefrog213,423
101.7%
gin750,780vsgo752,853
99.7%
fintrospect548,336vsfinagle556,888
98.5%
vertx-web2,411,118vsvertx2,503,741
96.3%
actframework2,424,671vsundertow2,544,955
95.3%
kami686,682vsgo752,853
91.2%
undertow-jersey-c3p0154,758vsundertow-jersey-hika170,106
91.0%
tornado-py355,282vstornado61,738
89.5%
finch477,543vsfinagle556,888
85.8%
beego604,659vsgo752,853
80.3%
falcon695,897vswsgi1,229,303
56.6%
falcon-py3610,525vswsgi1,229,303
49.7%
play2-scala221,514vsakka-http490,038
45.2%
falcore331,164vsgo752,853
44.0%
jetty305,109vsservlet730,095
41.8%
bottle462,141vswsgi1,229,303
37.6%
wicket271,565vsservlet730,095
37.2%
jetty-servlet262,519vsservlet730,095
36.0%
bottle-py3406,822vswsgi1,229,303
33.1%
jawn219,577vsservlet730,095
30.1%
weppy365,421vswsgi1,229,303
29.7%
weppy-py3363,672vswsgi1,229,303
29.6%
pyramid-py2331,732vswsgi1,229,303
27.0%
pyramid310,715vswsgi1,229,303
25.3%
finagle556,888vsnetty2,447,315
22.8%
tornado-postgresql-r13,267vstornado61,738
21.5%
restexpress497,486vsnetty2,447,315
20.3%
echo150,523vsgo752,853
20.0%
tapestry136,018vsservlet730,095
18.6%
falcon-pypy2221,939vswsgi1,229,303
18.1%
spring131,249vsservlet730,095
18.0%
activeweb129,324vsservlet730,095
17.7%
curacao119,759vsservlet730,095
16.4%
revel117,584vsgo752,853
15.6%
play270,167vsakka-http490,038
14.3%
django-py3159,900vswsgi1,229,303
13.0%
dropwizard94,713vsservlet730,095
13.0%
jooby294,958vsnetty2,447,315
12.1%
flask-py3121,901vswsgi1,229,303
9.9%
flask113,578vswsgi1,229,303
9.2%
play2-java-netty223,814vsnetty2,447,315
9.1%
bottle-pypy295,722vswsgi1,229,303
7.8%
play2-scala-scala-ne154,544vsnetty2,447,315
6.3%
django77,134vswsgi1,229,303
6.3%
aah44,783vsgo752,853
5.9%
ninja-standalone13,069vsjetty305,109
4.3%
web2py-optimized50,373vswsgi1,229,303
4.1%
goji22,435vsgo752,853
3.0%
weppy-pypy222,764vswsgi1,229,303
1.9%
web2py20,143vswsgi1,229,303
1.6%
scruffy41,366vsundertow2,544,955
1.6%
flask-pypy211,316vswsgi1,229,303
0.9%
flask-nginx-uwsgi4,868vswsgi1,229,303
0.4%
weppy-nginx-uwsgi4,752vswsgi1,229,303
0.4%
bottle-nginx-uwsgi4,735vswsgi1,229,303
0.4%
spark2,604vsservlet730,095
0.4%
Best (bar chart)
Data table
Latency
Framework overhead
Best (bar chart)
Data table
Latency
Framework overhead

Requirements summary

In this test, the framework responds with the simplest of responses: a "Hello, World" message rendered as plain text. The size of the response is kept small so that gigabit Ethernet is not the limiting factor for all implementations. HTTP pipelining is enabled and higher client-side concurrency levels are used for this test (see the "Data table" view).

Example response:

HTTP/1.1 200 OK Content-Length: 15 Content-Type: text/plain; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT Hello, World!

For a more detailed description of the requirements, see the Source Code and Requirements section.

Comments

If you have any comments about Round 15, please post at the Framework Benchmarks Google Group.

Framework details

Author / Sponsor
Home
Framework Source
Test Source
About
JSON
Single query
Multiple query
Fortunes
Updates
Plaintext
About
JSON
Single query
Multiple query
Fortunes
Updates
Plaintext

Results testing

Loading results for run-id from the TFB Results Dashboard...
Unable to load results for run-id from the TFB Results Dashboard.

Running these benchmarks in your own test environment? You can visualize the results by copying and pasting the contents of your results.json file in the text box below.

Test duration: seconds
Visualize results

Limitations

Note: This results web site can only render results for the frameworks it knows about by way of the metadata available for the currently-public Round. Therefore, results for frameworks that are not known by the metadata will not be visible.

Introduction

This is a performance comparison of many web application frameworks executing fundamental tasks such as JSON serialization, database access, and server-side template composition. Each framework is operating in a realistic production configuration. Results are captured on cloud instances and on physical hardware. The test implementations are largely community-contributed and all source is available at the GitHub repository.

Note: We use the word "framework" loosely to refer to platforms, micro-frameworks, and full-stack frameworks.

In a March 2013 blog entry, we published the results of comparing the performance of several web application frameworks executing simple but representative tasks: serializing JSON objects and querying databases. Since then, community input has been tremendous. We—speaking now for all contributors to the project—have been regularly updating the test implementations, expanding coverage, and capturing results in semi-regular updates that we call "rounds."

Results

View the latest results from Round 15. Or check out the previous rounds.

Making improvements

We expect that all frameworks' tests could be improved with community input. For that reason, we are extremely happy to receive pull requests from fans of any framework. We would like our tests for every framework to perform optimally, so we invite you to please join in.

What's to come

Feedback has been continuous and we plan to keep updating the project in several ways, such as:

  • Coverage of more frameworks. Thanks to community contributions to-date, the number of frameworks covered has already grown quite large. We're happy to add more if you submit a pull request.
  • Additional test types.
  • Tests on more types of hardware.
  • Enhancements to this results web site.

Additional resources

FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).

Current and Previous Rounds

Round 15 The project exceeded 3,000 stars on GitHub and has processed nearly 2,500 pull requests. Continuous benchmarking results are now available on the Results dashboard.

Round 14 Adoption of the mention-bot from Facebook has proven useful in notifying project participants of changes to their contributions. Continuous benchmarking provided a means for several community previews in this round, and we expect that to continue going forward. Note that this round was conducted only on physical hardware within the ServerCentral environment; tests on the cloud environment will return for Round 15.

Round 13 Microsoft's ASP.NET team delivers the most impressive improvement we've seen in this project—a 85,000% increase in plaintext results for ASP.NET Core—making it a top-performing framework at the fundamentals of HTTP request routing. Round 13 also sees new hardware and cloud environments from ServerCentral and Microsoft Azure.

Round 12 Marking the last round on the Peak environment, Round 12 sees some especially high Plaintext scores.

Round 11 26 more frameworks, three more languages, and the volume cranked to 11.

Round 10 Significant restructuring of the project's infrastructure, including re-organization of the project's directory structure and integration with Travis CI for rapid review of pull requests, and the addition of numerous frameworks.

Round 9 Thanks to the contribution of a 10-gigabit testing environment by Peak Hosting, the network barrier that frustrated top-performing frameworks in previous rounds has been removed. The Dell R720xd servers in this new environment feature dual Xeon E5-2660 v2 processors and illustrate how the spectrum of frameworks scale to forty processor cores.

Round 8 Six more frameworks contributed by the community takes the total count to 90 frameworks and 230 permutations (variations of configuration). Meanwhile, several implementations have been updated and the highest-performance platforms jockey for the top spot on each test's charts.

Round 7 After a several month hiatus, another large batch of frameworks have been added by the community. Even after consolidating a few, Round 7 counts 84 frameworks and over 200 test permutations! This round also was the first to use a community-review process. Future rounds will see roughly one week of preview and review by the community prior to release to the public here.

Round 6 Still more tests were contributed by the developer community, bringing the number of frameworks to 74! Round 6 also introduces an "plaintext" test type that exercises HTTP pipelining and higher client-side concurrency levels.

Round 5 The developer community comes through with the addition of ASP.NET tests ready to run on Windows. This round is the first with Windows tests, and we seek assistance from Windows experts to apply additional tuning to bring the results to parity with the Linux tests. Round 5 also introduces an "update" test type to exercise ORM and database writes.

Round 4 With 57 frameworks in the benchmark suite, we've added a filter control allowing you to narrow your view to only the frameworks you want to see. Round 4 also introduces the "Fortune" test to exercise server-side templates and collections.

Round 3 We created this stand-alone site for comparing the results data captured across many web application frameworks. Even more frameworks have been contributed by the community and the testing methodology was changed slightly thanks to enhancements to the testing tool named Wrk.

Round 2 In April, we published a follow-up blog entry named "Frameworks Round 2" where we incorporated changes suggested and contributed by the community.

Round 1 In a March 2013 blog entry, we published the results of comparing the performance of several web application frameworks executing simple but representative tasks: serializing JSON objects and querying databases. The community reaction was terrific. We are flattered by the volume of feedback. We received dozens of comments, suggestions, questions, criticisms, and most importantly, GitHub pull requests at the repository we set up for this project.

Unofficial Results

We operate a continuously-running benchmarking environment. You can see unofficial results as they are collected at the TFB Results Dashboard.

Motivation

Choosing a web application framework involves evaluation of many factors. While comparatively easy to measure, performance is frequently given little consideration. We hope to help change that.

Application performance can be directly mapped to hosting dollars, and for companies both large and small, hosting costs can be a pain point. Weak performance can also cause premature and costly scale pain by requiring earlier optimization efforts and increased architectural complexity. Finally, slow applications yield poor user experience and may suffer penalties levied by search engines.

What if building an application on one framework meant that at the very best your hardware is suitable for one tenth as much load as it would be had you chosen a different framework? The differences aren't always that extreme, but in some cases, they might be. Especially with several modern high-performance frameworks offering respectable developer efficiency, it's worth knowing what you're getting into.

Terminology

framework
We use the word framework loosely to refer to any HTTP server implementation upon which you could build a web application—a full-stack framework, a micro-framework, or even a web platform such as Rack, Servlet, or plain PHP.
platform
For us, platforms are broadly defined as anything situated between the programming language and the web framework (examples are Servlet, Netty, and Rack). By comparison to a full-stack framework or micro-framework, a platform may include a bare-bones HTTP server implementation with rudimentary request routing and virtually none of the higher-order functionality of frameworks such as form validation, input sanitization, templating, JSON serialization, and database connectivity. Frameworks are often built on top of platforms. To be thorough, and to compute framework overhead, we test several platforms as if they were frameworks.
permutation
A combination of attributes that compose a full technology stack being tested (take node.js for example, we might have one permutation with MongoDB and another with MySQL). Some frameworks have seen many permutations contributed by the community; others only one or few.
test type
One of the workloads we exercise, such as JSON serialization, single-query, multiple-query, fortunes, data updates, and plaintext.
test
An individual test is a measurement of the performance of a permutation's implementation of a test type. For example, a test might be measuring Wicket paired with MySQL running the single-query test type.
implementation
Sometimes called "test implementations," these are the bodies of code and configuration created to test permutations according to the requirements. These are frequently contributed by fans, advocates, or the maintainers of frameworks. Together with the toolset, test implementations are the meat of this project.
toolset
A set of Python scripts that run our tests.
run
An execution of the benchmark toolset across the suite of test implementations, either in full or in part, in order to capture results for any purpose.
preview
A capture of data from a run used by project participants to sanity-check prior to an official round.
round
A posting of "official" results on this web site. This is mostly for ease of consumption by readers and good-spirited & healthy competitive bragging rights. For in-depth analysis, we encourage you to examine the source code and run the tests on your own hardware.

Expected questions

We expect that you might have a bunch of questions. Here are some that we're anticipating. But please contact us if you have a question we're not dealing with here or just want to tell us we're doing it wrong.

Frameworks and configuration

  1. "You call x a framework, but it's a platform." See the terminology section above. We are using the word "framework" loosely to refer to anything found on the spectrum ranging from full-stack frameworks, micro-frameworks, to platforms. If it's used to build web applications, it probably qualifies. That said, we understand that comparing a full-stack framework versus platforms or vice-versa is unusual. We feel it's valuable to be able to compare these, for example to understand the performance overhead of additional abstraction. You can use the filters in the results viewer to adjust the rows you see in the charts.
  2. "You configured framework x incorrectly, and that explains the numbers you're seeing." Whoops! Please let us know how we can fix it, or submit a GitHub pull request, so we can get it right.
  3. "Why include this Gemini framework I've never heard of?" We have included our in-house Java web framework, Gemini, in our tests. We've done so because it's of interest to us. You can consider it a stand-in for any relatively lightweight minimal-locking Java framework. While we're proud of how it performs among the well-established field, this exercise is not about Gemini.
  4. "Why don't you test framework X?" We'd love to, if we can find the time. Even better, craft the test implementation yourself and submit a GitHub pull request so we can get it in there faster!
  5. "Some frameworks use process-level concurrency; have you accounted for that?" Yes, we've attempted to use production-grade configuration settings for all frameworks, including those that rely on process-level concurrency. For the EC2 tests, for example, such frameworks are configured to utilize the two virtual cores provided on an c3.large (in previous rounds, m1.large) instance. For the i7 tests, they are configured to use the eight hyper-threading cores of our hardware's i7 CPUs.
  6. "Have you enabled APC for the PHP tests?" Yes, the PHP tests run with APC and PHP-FPM on nginx.
  7. "Why are you using a (slightly) old version of framework X?" It's nothing personal! With so many frameworks we have a never-ending game of whack-a-mole. If you think an update will affect the results, please let us know (or better yet, submit a GitHub pull request) and we'll get it updated!
  8. "It's unfair and possibly even incorrect to compare X and Y!" It may be alarming at first to see the full results table, where one may evaluate frameworks vs platforms; MySQL vs Postgres; Go vs Python; ORM vs raw database connectivity; and any number of other possibly irrational comparisons. Many readers desire the ability to compare these and other permutations. If you prefer to view an unpolluted subset, you may use the filters available at the top of the results page. We believe that comparing frameworks with plausible and diverse technology stacks, despite the number of variables, is precisely the value of this project. With sufficient time and effort, we hope to continuously broaden the test permutations. But we recommend against ignoring the data on the basis of concerns about multi-variable comparisons. Read more opinion on this at Brian Hauer's personal blog.
  9. "If you are testing production deployments, why is logging disabled?" At present, we have elected to run tests with logging features disabled. Although this is not consistent with production deployments, we avoid a few complications related to logging, most notably disk capacity and consistent granularity of logging across all test implementations. In spot tests, we have not observed significant performance impact from logging when enabled. If there is strong community consensus that logging is necessary, we will reconsider this.
  10. "Tell me about the Windows configuration." We are very thankful to the community members who have contributed Windows tests. In fact, nearly the entirety of the Windows configuration has been contributed by subject-matter experts from the community. Thanks to their effort, we now have tests covering both Windows paired with Linux databases and Windows paired with Microsoft SQL Server. As with all aspects of this project, we welcome continued input and tuning by other experts. If you have advice on better tuning the Windows tests, please submit GitHub issues or pull requests.

The tests

  1. "Framework X has in-memory caching, why don't you use that?" In-memory caching, as provided by some frameworks, yields higher performance than repeatedly hitting a database, but isn't available in all frameworks, so we omitted in-memory caching from these tests. Cache tests are planned for later rounds.
  2. "What about other caching approaches, then?" Remote-memory or near-memory caching, as provided by Memcached and similar solutions, also improves performance and we would like to conduct future tests simulating a more expensive query operation versus Memcached. However, curiously, in spot tests, some frameworks paired with Memcached were conspicuously slower than other frameworks directly querying the authoritative MySQL database (recognizing, of course, that MySQL had its entire data-set in its own memory cache). For simple "get row ID n" and "get all rows" style fetches, a fast framework paired with MySQL may be faster and easier to work with versus a slow framework paired with Memcached.
  3. "Why doesn't your test include more substantial algorithmic work?" Great suggestion. We hope to in the future!
  4. "What about reverse proxy options such as Varnish?" We are expressly not using reverse proxies on this project. There are other benchmark projects that evaluate the performance of reverse proxy software. This project measures the performance of web applications in any scenario where requests reach the application server. Given that objective, allowing the web application to avoid doing the work thanks to a reverse proxy would invalidate the results. If it's difficult to conceptualize the value of measuring performance beyond the reverse proxy, imagine a scenario where every response provides user-specific and varying data. It's also notable that some platforms respond with sufficient performance to potentially render a reverse proxy unnecessary.
  5. "Do all the database tests use connection pooling?" Yes, our expectation is that all tests use connection pooling.
  6. "How is each test run?" Each test is executed as follows:
    1. Restart the database servers.
    2. Start the platform and framework using their start-up mechanisms.
    3. Run a 5-second primer at 8 client-concurrency to verify that the server is in fact running. These results are not captured.
    4. Run a 15-second warmup at 256 client-concurrency to allow lazy-initialization to execute and just-in-time compilation to run. These results are not captured.
    5. Run a 15-second captured test for each of the concurrency levels (or iteration counts) exercised by the test type. Concurrency-variable test types are tested at 8, 16, 32, 64, 128, and 256 client-side concurrency. The high-concurrency plaintext test type is tested at 256, 1,024, 4,096, and 16,384 client-side concurrency.
    6. Stop the platform and framework.
  7. "Hold on, 15 seconds is not enough to gather useful data." This is a reasonable concern. But in examining the data, we have seen no evidence that the results have changed by reducing the individual test durations from 60 seconds to 15 seconds. The duration reduction was made necessary by the growing number of test permutations and a target that the full suite complete in less than one day. With additional effort, we aim to build a continuously-running test environment that will pull the latest source and begin a new run as soon as a previous run completes. When we have such an environment ready, we will be comfortable with multi-day execution times, so we plan to extend the duration of each test when that happens.
  8. "Also, a 15-second warmup is not sufficient." On the contrary, we have not yet seen evidence suggesting that any additional warmup time is beneficial to any framework. In fact, for frameworks based on JIT platforms such as the Java Virtual Machine (JVM), spot tests show that the JIT has even completed its work already after just the primer and before the warmup starts—the warmup (256-concurrency) and real 256-concurrency tests yield results that are separated only by test noise. However, as with test durations, we intend to increase the duration of the warmup when we have a continuously-running test environment.

Environment

  1. "What is Wrk?" Although many web performance tests use ApacheBench from Apache to generate HTTP requests, we now use Wrk for this project. ApacheBench remains a single-threaded tool, meaning that for higher-performance test scenarios, ApacheBench itself is a limiting factor. Wrk is a multithreaded tool that provides a similar function, allowing tests to run for a prescribed amount of time (rather than limited to a number of requests) and providing us result data including total requests completed and latency information.
  2. "Doesn't benchmarking on Amazon EC2 invalidate the results?" Our opinion is that doing so confirms precisely what we're trying to test: performance of web applications within realistic production environments. Selecting EC2 as a platform also allows the tests to be readily verified by anyone interested in doing so. However, we've also executed tests on our Core i7 (Sandy Bridge) workstations running Ubuntu as a non-virtualized comparison. Doing so confirmed our suspicion that the ranked order and relative performance across frameworks is mostly consistent between EC2 and physical hardware. That is, while the EC2 instances were slower than the physical hardware, they were slower by roughly the same proportion across the spectrum of frameworks.
  3. "Tell me about your physical hardware." For the tests we refer to as "i7" tests, we're using our office workstations. These use Intel i7-2600K processors, making them a little antiquated, to be honest. These are connected via an unmanaged low-cost gigabit Ethernet switch. In previous rounds, we used a two-machine configuration where the load-generation and database role coexisted. Although these two roles were not crowding one another out (neither role was starved for CPU time), as of Round 7, we are using a three-machine configuration for the physical hardware tests. The machine roles are:
    • Application server, which hosts the application code and web server, where applicable.
    • Database server, which hosts the common databases. Starting with Round 5, we equipped the database server with a Samsung 840 Pro SSD.
    • Load generator, which makes HTTP requests to the Application server via the Wrk load generation tool.
  4. "What is Resin? Why aren't you using Tomcat for the Java frameworks?" Resin is a Java application server. The GPL version that we used for our tests is a relatively lightweight Servlet container. We tested on Tomcat as well but ultimately dropped Tomcat from our tests because Resin was slightly faster across all Servlet-based frameworks.
  5. "Do you run any warmups before collecting results data?" Yes. See "how is each test run" above. Every test is preceded by a warmup and brief (several seconds) cooldown prior to gathering test data.

Results

  1. "I am about to start a new web application project; how should I interpret these results?" Most importantly, recognize that performance data should be one part of your decision-making process. High-performance web applications reduce hosting costs and improve user experience. Additionally, recognize that while we have aimed to select test types that represent workloads that are common for web applications, nothing beats conducting performance tests yourself for the specific workload of your application. In addition to performance, consider other requirements such as your language and platform preference; your invested knowledge in one or more of the frameworks we've tested; and the documentation and support provided by the framework's community. Combined with an examination of the source code, the results seen here should help you identify a platform and framework that is high-performance while still meeting your other requirements.
  2. "Why are the leaderboards for JSON Serialization and Plaintext so different on EC2 versus i7?" Put briefly, for fast frameworks on our i7 physical hardware, the limiting factor for the JSON test is our gigabit Ethernet; whereas on EC2, the limit is the CPU. Assuming proper response headers are provided, at approximately 200,000 non-pipelined and 550,000 pipelined responses per second and above, the network is saturated.
  3. "Where did earlier rounds go?" To better capture HTTP errors reported by Wrk, we have restructured the format of our results.json file. The test tool changed at Round 2 and some framework IDs were changed at Round 3. As a result, the results.json for Rounds 1 and 2 would have required manual editing and we opted to simply remove the previous rounds from this site. You can still see those rounds at our blog: Round 1, Round 2.
  4. "What does 'Did not complete' mean?" Starting with Round 9, we have added validation checks to confirm that implementations are behaving as we have specified in the requirements section of this site. An implementation that does not return the correct results, bypasses some of the requirements, or even formats the results in a manner inconsistent with the requirements will be marked as "Did not complete." We have solicited corrections from prior contributors and have attempted to address many of these, but it will take more time for all implementations to be correct. If you are a project participant and your contribution is marked as "Did not complete," please help us resolve this by contacting us at the GitHub repository. We may ultimately need a pull request from you, but we'd be happy to help you understand what specifically is triggering a validation error with your implementation.
  5. "Why are Stripped test implementations hidden by default?" Since the introduction of Stripped test implementations, we have debated whether they should be included at all. A Stripped test implementation is one that is specially crafted to excel at our benchmark. By comparison, a "Realistic" test implementation should be demonstrative of the general-purpose, best-practices compliant, and production-class approach for the given framework. We have decided to hide Stripped tests by default because we feel that while their results have some value, that value is exceedingly low for the vast majority of consumers of the data. You may still view the results for Stripped tests by enabling the Stripped Implementation Approach in the filters control panel.
  6. "What exactly causes a test implementation to be classified as Stripped?" It's not possible to paint an exact picture, but conceptually, a Stripped implementation is characterized by being configured or engineered expressly to the requirements of our benchmark tests. By comparison, a Realistic implementation will use a production-grade configuration of a general-purpose web application framework that meets our requirements. When we first introduced the notion of Stripped, it referred to configurations of otherwise normal software that had been stripped of some normal behaviors. For example, removing some middleware from Rails or Django. However, we have broadened the definition of Stripped to also refer to bespoke software that has been crafted expressly to meet our test types' requirements.
  7. "I have collected results from my own test environment; can I visualize them in a manner similar to this web site?" Yes, use the test results visualization feature to visualize results you've gathered. Be aware that you need to specify the per-test duration setting (in seconds) and that only known frameworks will be rendered.

Contributions

  1. "Do you accept contributions?" Absolutely! Please visit the project's GitHub repository to join the project. In fact, the majority of the test implementations are community-contributed.
  2. "You emphasize production-grade software. Do you accept early builds of frameworks or toy projects?" Actually, yes, we are quite liberal with accepting contributions, including those that don't meet the target of production-grade. However, in many cases, such implementations will be marked as "Stripped," meaning they are not recommended for use in real-world production projects. They will still be measured and can be made visible by enabling Stripped in the filters control panel.
  3. "I am a contributor and my framework hasn't shown up in an official round yet. How do I know how my contribution will performn in your test environment?" We have a benchmark environment that is continuously running the benchmark suite, which we imaginatively named "Continuous Benchmarking™" (not actually trademarked). The continuous benchmark runner posts results to the TFB Results Dashboard. If you have made a contribution, you should see its results posted to the dashboard within a few days to a week, when the next regular continuous run completes.

Join the conversation

FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).

Simulating production environments

We aim to configure every framework according to the best practices for production deployments gleaned from documentation and popular community opinion, and we ask contributors to apply the same rule of thumb. We want each test implementation (see "Terminology" section) to approximate a sensible production deployment as accurately as possible. We also want this project to be as transparent as possible, so we have posted our test suites on GitHub.

Environment details

This project measures performance in two common deployment scenarios: cloud instances and physical hardware. To-date, each round has used a single representative environment for each of these scenarios. The particular specifications of the environments have evolved over time as shown below.

Cloud environments

Azure (rounds 13 onward)
Microsoft Azure D3v2 instances; switched gigabit Ethernet.
AWS (rounds 1 through 12)
Amazon EC2 c3.large instances (2 vCPU each); switched gigabit Ethernet (m1.large was used through Round 9).

Physical hardware environments

Citrine (rounds 16 onward)
Three homogeneous Dell R440 servers each equipped with an Intel Xeon Gold 5120 CPU, 32 GB of memory, and an enterprise SSD. Dedicated Cisco 10-gigabit Ethernet switch. Provided by Microsoft.
ServerCentral (rounds 13 through 15)
Dell R910 (4x 10-Core Intel Xeon E7-4850 CPUs) application server; Dell R710 (2x 4-Core Intel Xeon E5520 CPUs) database server; switched 10-gigabit Ethernet. Provided by ServerCentral.
Peak (rounds 9 through 12)
Dell R720xd dual Intel Xeon E5-2660 v2 (40 HT cores) with 32 GB memory; database servers equipped with SSDs in RAID; switched 10-gigabit Ethernet. Provided by Peak Hosting.
i7 (rounds 1 through 8)
In-house Intel Sandy Bridge Core i7-2600K workstations with 8 GB memory (early 2011 vintage); database server equipped with Samsung 840 Pro SSD; switched gigabit Ethernet.

General resources

If you are interested in this project, we'd love to hear from you.

FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).
FAQ Frequently asked questions and the story behind why we do this.
GitHub Code for the project at GitHub. Pull Requests welcomed!
Google Group Mailing list for questions, comments, recommendations, criticisms, or any other form of feedback.
@TFBenchmarks Tweet at us or follow us on Twitter for quick updates.
TFB-Status Continuous Benchmarking results (most useful for contributors).

Test requirements

We invite fans of frameworks and especially authors or maintainers of frameworks to join us in expanding the coverage of this project by implementing tests and contributing to the GitHub repository. The following are specifications for each of the test types we have included to-date in this project. Do not be alarmed; the specifications read quite verbose, but that's because they are specifications. The implementations tend to be quite easy in practice.

This project is evolving and we will periodically add new test types. As new test types are added, we encourage but do not require contributors of previous implementations to implement tests for the new test types. Wholly new test implementations are also encouraged to include all test types but are not required to do so. If you have limited time, we recommend you start with the easiest test types (1, 2, 3, and 6) and then continue beyond those as time permits.

General requirements

The following requirements apply to all test types below.

  1. All test implementations should be production-grade. The particulars of this will vary by framework and platform, but the general sentiment is that the code and configuration should be suitable for a production deployment. The word should is used here because production-grade is our goal, but we don't want this to be a roadblock. If you're submitting a new test and uncertain whether your code is production-grade, submit it anyway and then solicit input from other subject-matter experts.
  2. This project intends to measure performance of production-grade deployments of real-world web application frameworks and platforms. All implementations are expected (but not required) to be based on robust implementations of the HTTP protocol. Implementations that are not based on a realistic HTTP implementation will be marked as Stripped (see Questions section for more detail on the Stripped classification). Rather than stipulate a rigid specification of HTTP requirements, we defer to common sense: the implementation should provide a viable subset of HTTP to reasonably unbiased outsiders familiar with HTTP.
  3. All test implementations must disable all disk logging. For many reasons, we expect all tests will run without writing logs to disk. Most importantly, the volume of requests is sufficiently high to fill up disks even with only a single line written to disk per request. Please disable all forms of disk logging. We recommend but do not require disabling console logging as well.
  4. Specific characters and character case matter. Assume the client consuming your service's JSON responses will be using a case-sensitive language such as JavaScript. In other words, if a test specifies that a map's key is id, use id. Do not use Id or ID. This strictness is required not only because it's sensible but also because our automated validation checks are picky.
  5. All test types require Server and Date HTTP response headers. We expect the Server header to be whatever is normal for the platform or framework. If the framework does not normally provide a Server response header, we nevertheless require that you provide one as this roughly normalizes network load across all implementations. For Date, we expect that the rendered date be accurate. However, it does not need to be rendered from the system clock to a byte buffer for each request. Re-rendering once per second is an acceptable tactical optimization (and is an optimization baked into many frameworks).
  6. Some form of request routing is required, even if only a single test type (e.g., Plaintext) is implemented. In most cases, the framework's canonical router or a mainstream router library will be used. In some cases, it is considered normal and sufficiently production-grade to use hand-crafted minimalist routing using control structures such as if/else branching. This is acceptable where it is considered canonical for the framework.
  7. Except where noted, all database queries should be delivered to the database servers as-is and not coalesced or deduplicated at the database driver. In all cases where a database query is required, it is expected that the query will reach and execute on the database server—that is, the full round-trip effort of a query will be part of the workload.

Test type 1: JSON serialization

This test exercises the framework fundamentals including keep-alive support, request routing, request header parsing, object instantiation, JSON serialization, response header generation, and request count throughput.

Requirements

  1. For each request, an object mapping the key message to Hello, World! must be instantiated.
  2. The recommended URI is /json.
  3. A JSON serializer must be used to convert the object to JSON.
  4. The response text must be {"message":"Hello, World!"}, but white-space variations are acceptable.
  5. The response content length should be approximately 28 bytes.
  6. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
  7. The response headers must include either Content-Length or Transfer-Encoding.
  8. The response headers must include Server and Date.
  9. gzip compression is not permitted.
  10. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  11. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  12. The request handler will be exercised at concurrency levels ranging from 8 to 256.
  13. The request handler will be exercised using GET requests.

Example request

GET /json HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Type: application/json Content-Length: 28 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"message":"Hello, World!"}

Test type 2: Single database query

This test exercises the framework's object-relational mapper (ORM), random number generator, database driver, and database connection pool.

Requirements

  1. For every request, a single row from a World table must be retrieved from a database table.
  2. The recommended URI is /db.
  3. The schema for World is id (int, primary key) and randomNumber (int), except for MongoDB, wherein the identity column is _id, with the leading underscore.
  4. The World table is known to contain 10,000 rows.
  5. The row retrieved must be selected by its id using a random number generator (ids range from 1 to 10,000).
  6. The row should be converted to an object using an object-relational mapping (ORM) tool. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
  7. The object (or database row, if an ORM is not used) must be serialized to JSON.
  8. The response content length should be approximately 32 bytes.
  9. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
  10. The response headers must include either Content-Length or Transfer-Encoding.
  11. The response headers must include Server and Date.
  12. Use of an in-memory cache of World objects or rows by the application is not permitted.
  13. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
  14. gzip compression is not permitted.
  15. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  16. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  17. The request handler will be exercised at concurrency levels ranging from 8 to 256.
  18. The request handler will be exercised using GET requests.

Example request

GET /db HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 32 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT {"id":3217,"randomNumber":2149}

Test type 3: Multiple database queries

This test is a variation of Test #2 and also uses the World table. Multiple rows are fetched to more dramatically punish the database driver and connection pool. At the highest queries-per-request tested (20), this test demonstrates all frameworks' convergence toward zero requests-per-second as database activity increases.

Requirements

  1. For every request, an integer query string parameter named queries must be retrieved from the request. The parameter specifies the number of database queries to execute in preparing the HTTP response (see below).
  2. The recommended URI is /queries.
  3. The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
  4. The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
  5. Each row must be selected randomly in the same fashion as the single database query test (Test #2 above).
  6. This test is designed to exercise multiple queries, each requiring a round-trip to the database server, and with each resulting row selected individually. It is not acceptable to use batches. It is not acceptable to execute multiple SELECTs within a single statement. It is not acceptable to retrieve all required rows using a SELECT ... WHERE id IN (...) clause.
  7. Each World object must be added to a list or array.
  8. The list or array must be serialized to JSON and sent as a response.
  9. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
  10. The response headers must include either Content-Length or Transfer-Encoding.
  11. The response headers must include Server and Date.
  12. Use of an in-memory cache of World objects or rows by the application is not permitted.
  13. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
  14. gzip compression is not permitted.
  15. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  16. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  17. The request handler will be exercised at 256 concurrency only.
  18. The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
  19. The request handler will be exercised using GET requests.

Example request

GET /queries?queries=10 HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 315 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT [{"id":4174,"randomNumber":331},{"id":51,"randomNumber":6544},{"id":4462,"randomNumber":952},{"id":2221,"randomNumber":532},{"id":9276,"randomNumber":3097},{"id":3056,"randomNumber":7293},{"id":6964,"randomNumber":620},{"id":675,"randomNumber":6601},{"id":8414,"randomNumber":6569},{"id":2753,"randomNumber":4065}]

Test type 4: Fortunes

This test exercises the ORM, database connectivity, dynamic-size collections, sorting, server-side templates, XSS countermeasures, and character encoding.

Requirements

  1. The recommended URI is /fortunes.
  2. A Fortune database table contains a dozen Unix-style fortune-cookie messages.
  3. The schema for Fortune is id (int, primary key) and message (varchar), except for MongoDB, wherein the identity column is _id, with the leading underscore.
  4. Using an ORM, all Fortune objects must be fetched from the Fortune table, and placed into a list data structure. Tests that do not use an ORM will be classified as "raw" meaning they use the platform's raw database connectivity.
  5. Each request to the Fortunes URI should result in exactly one database query as specified above. Query coalescing or deduplication is not permitted. See the General requirements section above. Conceptually, the expectation is that after measuring an implementation of the Fortunes test type, the aggregate number of queries executed by the database server during the test is equal to the number of Fortunes HTTP requests completed.
  6. The list data structure must be a dynamic-size or equivalent and should not be dimensioned using foreknowledge of the row-count of the database table.
  7. Within the scope of the request, a new Fortune object must be constructed and added to the list. This confirms that the data structure is dynamic-sized. The new fortune is not persisted to the database; it is ephemeral for the scope of the request.
  8. The new Fortune's message must be "Additional fortune added at request time."
  9. The list of Fortune objects must be sorted by the order of the message field. No ORDER BY clause is permitted in the database query (ordering within the query would be of negligible value anyway since a newly instantiated Fortune is added to the list prior to sorting).
  10. The sorted list must be provided to a server-side template and rendered to simple HTML (see below for minimum template). The resulting HTML table displays each Fortune's id number and message text.
  11. This test does not include external assets (CSS, JavaScript); a later test type will include assets.
  12. The HTML generated by the template must be sent as a response.
  13. Be aware that the message text fields are stored as UTF-8 and one of the fortune cookie messages is in Japanese.
  14. The resulting HTML must be delivered using UTF-8 encoding.
  15. The Japanese fortune cookie message must be displayed correctly.
  16. Be aware that at least one of the message text fields includes a <script> tag.
  17. The server-side template must assume the message text cannot be trusted and must escape the message text properly.
  18. The implementation is encouraged to use best practices for templates such as layout inheritence, separate header and footer files, and so on. However, this is not required. We request that implementations do not manage assets (JavaScript, CSS, images). We are deferring asset management until we can craft a more suitable test.
  19. The response content type must be set to text/html.
  20. The response headers must include either Content-Length or Transfer-Encoding.
  21. The response headers must include Server and Date.
  22. Use of an in-memory cache of Fortune objects or rows by the application is not permitted.
  23. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
  24. gzip compression is not permitted.
  25. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  26. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  27. The request handler will be exercised at concurrency levels ranging from 8 to 256.
  28. The request handler will be exercised using GET requests.

Example request

GET /fortunes HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 1196 Content-Type: text/html; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT <!DOCTYPE html><html><head><title>Fortunes</title></head><body><table><tr><th>id</th><th>message</th></tr><tr><td>11</td><td>&lt;script&gt;alert(&quot;This should not be displayed in a browser alert box.&quot;);&lt;/script&gt;</td></tr><tr><td>4</td><td>A bad random number generator: 1, 1, 1, 1, 1, 4.33e+67, 1, 1, 1</td></tr><tr><td>5</td><td>A computer program does what you tell it to do, not what you want it to do.</td></tr><tr><td>2</td><td>A computer scientist is someone who fixes things that aren&apos;t broken.</td></tr><tr><td>8</td><td>A list is only as strong as its weakest link. — Donald Knuth</td></tr><tr><td>0</td><td>Additional fortune added at request time.</td></tr><tr><td>3</td><td>After enough decimal places, nobody gives a damn.</td></tr><tr><td>7</td><td>Any program that runs right is obsolete.</td></tr><tr><td>10</td><td>Computers make very fast, very accurate mistakes.</td></tr><tr><td>6</td><td>Emacs is a nice operating system, but I prefer UNIX. — Tom Christaensen</td></tr><tr><td>9</td><td>Feature: A bug with seniority.</td></tr><tr><td>1</td><td>fortune: No such file or directory</td></tr><tr><td>12</td><td>フレームワークのベンチマーク</td></tr></table></body></html>

Minimum template

Along with the example response above, the following Mustache template illustrates the minimum requirements for the server-side template. White-space can be optionally eliminated.

<!DOCTYPE html> <html> <head><title>Fortunes</title></head> <body> <table> <tr><th>id</th><th>message</th></tr> {{#.}} <tr><td>{{id}}</td><td>{{message}}</td></tr> {{/.}} </table> </body> </html>

Test type 5: Database updates

This test is a variation of Test #3 that exercises the ORM's persistence of objects and the database driver's performance at running UPDATE statements or similar. The spirit of this test is to exercise a variable number of read-then-write style database operations.

Requirements

  1. The recommended URI is /updates.
  2. For every request, an integer query string parameter named queries must be retrieved from the request. The parameter specifies the number of rows to fetch and update in preparing the HTTP response (see below).
  3. The queries parameter must be bounded to between 1 and 500. If the parameter is missing, is not an integer, or is an integer less than 1, the value should be interpreted as 1; if greater than 500, the value should be interpreted as 500.
  4. The request handler must retrieve a set of World objects, equal in count to the queries parameter, from the World database table.
  5. Each row must be selected randomly using one query in the same fashion as the single database query test (Test #2 above). As with the read-only multiple-query test type (#3 above), use of IN clauses or similar means to consolidate multiple queries into one operation is not permitted. Similarly, use of a batch or multiple SELECTs within a single statement are not permitted.
  6. At least the randomNumber field must be read from the database result set.
  7. Each World object must have its randomNumber field updated to a new random integer between 1 and 10000.
  8. Each World object must be persisted to the database with its new randomNumber value.
  9. Use of batch updates is acceptable but not required. To be clear: batches are not permissible for selecting/reading the rows, but batches are acceptable for writing the updates.
  10. Use of transactions is acceptable but not required. If transactions are used, a transaction should only encapsulate a single iteration, composed of a single read and single write. Transactions should not be used to consolidate multiple iterations into a single operation.
  11. All updates should be completed prior to returning the HTTP response.
  12. For raw tests (that is, tests without an ORM), each updated row must receive a unique new randomNumber value. It is not acceptable to change the randomNumber value of all rows to the same random number using an UPDATE ... WHERE id IN (...) clause.
  13. Each World object must be added to a list or array.
  14. The list or array must be serialized to JSON and sent as a response.
  15. The response content type must be set to application/json. Note that specifying the character encoding is unnecessary for this content type.
  16. The response headers must include either Content-Length or Transfer-Encoding.
  17. The response headers must include Server and Date.
  18. Use of an in-memory cache of World objects or rows by the application is not permitted.
  19. Use of prepared statements for SQL database tests (e.g., for MySQL) is encouraged but not required.
  20. gzip compression is not permitted.
  21. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  22. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  23. The request handler will be exercised at 256 concurrency only.
  24. The request handler will be exercised with query counts of 1, 5, 10, 15, and 20.
  25. The request handler will be exercised using GET requests.

Example request

GET /updates?queries=10 HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 315 Content-Type: application/json Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT [{"id":4174,"randomNumber":331},{"id":51,"randomNumber":6544},{"id":4462,"randomNumber":952},{"id":2221,"randomNumber":532},{"id":9276,"randomNumber":3097},{"id":3056,"randomNumber":7293},{"id":6964,"randomNumber":620},{"id":675,"randomNumber":6601},{"id":8414,"randomNumber":6569},{"id":2753,"randomNumber":4065}]

Test type 6: Plaintext

This test is an exercise of the request-routing fundamentals only, designed to demonstrate the capacity of high-performance platforms in particular. Requests will be sent using HTTP pipelining. The response payload is still small, meaning good performance is still necessary in order to saturate the gigabit Ethernet of the test environment.

The General Requirements section above applies to all test types, including Plaintext. Importantly, implementations should be production-grade deployments of real-world web application frameworks or platforms. The Plaintext test type is historically highly competitive, leading to some implementations being highly tuned to the express and narrow requirements identified below. We will usually accept such implementations, but they are subject to being classified as Stripped.

Requirements

  1. The recommended URI is /plaintext.
  2. The response content type must be set to text/plain.
  3. The response body must be Hello, World!.
  4. This test is not intended to exercise the allocation of memory or instantiation of objects. Therefore it is acceptable but not required to re-use a single buffer for the response text (Hello, World). However, the response must be fully composed from the response text and response headers within the scope of each request and it is not acceptable to store the entire payload of the response, or an unnaturally large subset of the response, headers inclusive, as a pre-rendered buffer. "Buffer" here refers to a byte array, byte buffer, character array, character buffer, string, or string-like data structure. The spirit of the test is to require the construction of the HTTP response as is typically done by a framework or platform via concatenation of strings or similar. For example, pre-rendering a buffer with HTTP/1.1 200 OK<cr>Content-length: 15<cr>Server: Example<cr> would not be acceptable.
  5. The response headers must include either Content-Length or Transfer-Encoding.
  6. The response headers must include Server and Date.
  7. gzip compression is not permitted.
  8. Server support for HTTP Keep-Alive is strongly encouraged but not required.
  9. Server support for HTTP/1.1 pipelining is assumed. Servers that do not support pipelining may be included but should downgrade gracefully. If you are unsure about your server's behavior with pipelining, test with the wrk load generation tool used in our tests.
  10. If HTTP Keep-Alive is enabled, no maximum Keep-Alive timeout is specified by this test.
  11. The request handler will be exercised at 256, 1024, 4096, and 16,384 concurrency.
  12. The request handler will be exercised using GET requests.

Example request

GET /plaintext HTTP/1.1 Host: server User-Agent: Mozilla/5.0 (X11; Linux x86_64) Gecko/20130501 Firefox/30.0 AppleWebKit/600.00 Chrome/30.0.0000.0 Trident/10.0 Safari/600.00 Cookie: uid=12345678901234567890; __utma=1.1234567890.1234567890.1234567890.1234567890.12; wd=2560x1600 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Connection: keep-alive

Example response

HTTP/1.1 200 OK Content-Length: 15 Content-Type: text/plain; charset=UTF-8 Server: Example Date: Wed, 17 Apr 2013 12:00:00 GMT Hello, World!